I built a similar setup a few years back for a busy web application (only I did it with Squid instead of Varnish), and it worked out well.
I would recommend using your first setup (HAProxy -> Varnish) with two modifications:
- Add a secondary HAProxy server using
keepalived
and a shared virtual IP - Use the
balance uri
load balancing algorithm to optimize cache hits
Pros:
- Peace of mind with HAProxy (x2) and Varnish (x3) redundancy
- Better hit rate efficiency on Varnish with HAProxy URI load balancing option
- Better performance from the cache servers as they don’t need to keep as much in memory
- Invalidating cache is easier since the same URI will go to the same server every time
Cons:
- URI balancing works well, but if a cache server goes down, your backend servers will get hit as the other cache server(s) that pick up the slack from the updated URI balancing hash will need to re-retrieve the cached data. Maybe not a big con, but I did have to keep that in mind for my system.