I don’t know why vim uses so many colors to highlight with, it doesn’t really help you figure out what’s going on.
I modified my colorscheme to only use one color to highlight (with another to show where theres a difference within a line) and it made all the difference.
Before

After

I did this by adding the following to the end of my colorscheme file (~/.vim/colors/mycolorscheme.vim).
highlight DiffAdd cterm=bold ctermfg=10 ctermbg=17 gui=none guifg=bg guibg=Red
highlight DiffDelete cterm=bold ctermfg=10 ctermbg=17 gui=none guifg=bg guibg=Red
highlight DiffChange cterm=bold ctermfg=10 ctermbg=17 gui=none guifg=bg guibg=Red
highlight DiffText cterm=bold ctermfg=10 ctermbg=88 gui=none guifg=bg guibg=Red
cterm– sets the stylectermfg– set the text colorctermbg– set the highlightingDiffAdd– line was addedDiffDelete– line was removedDiffChange– part of the line was changed (highlights the whole line)DiffText– the exact part of the line that changed
I used this link as a reference for the color numbers.
Note: I didn’t set the gui options because I use a different colorscheme for macvim/gvim