Get distinct values of Queryset by field

What you want is:

Visit.objects.filter(stuff).values("ip_address").annotate(n=models.Count("pk"))

What this does is get all ip_addresses and then it gets the count of primary keys (aka number of rows) for each ip address.