If I understand you correctly, the following should do the trick:
favorites = Favorite.objects.filter(user=request.user)
color_ids = favorites.values_list('item__color', flat=True).distinct()
colors = Color.objects.filter(id__in=color_ids)
There has to be a cleaner way than that though.
Edit: A much cleaner solution:
colors = Color.objects.filter(item__favorite__user=request.user).distinct()