Possible to do a delete with a HAVING clause?

Not really. The having clause implies an aggregation, which means you don’t have the original rows any more.

I think you want the following:

DELETE from UserPredictions
where UserId in (select UserId from UserPredictions group by UserId having count(*) < 500)