How do websocket connections work through a load balancer?

by

Load balancer and reverse proxy have diffrent use case.

  • Main use case of the load balancer is to distribute the load among node in a group of the server to manage the resource utilisation of each node

  • One of the use cases of a reverse proxy is to hide server meta information (ip,port etc..) from the client. It’s some sort of security.

We can configure the reverse proxy with load balancer or we can configure the reverse proxy alone as well.

Configuring the load balancer for a single node doesn’t make sense but we can configure the reverse proxy for a single node.

Handling the WebSocket load or distributing the load in websocket nodes (in a cluster) is a very complicated implementation because of its use case and limitation.

Why it’s complicated:

  1. WebSocket is sticky connections, once you connected you will remain connected as long as your application is active

  2. you have a limit to open WebSocket connections in the server (default is 63k). you can scale it more by some setting in kernal level, then you need to compromise with your system resources

Clarifying your doubt: if you put WebSocket behind the load balancer,
then WebSocket will communicate through a load balancer from the client and the client will communicate with a load balancer.

if two WebSocket connection will open (Assuming one node in load balancer) for each client request then there is no use of load balancer and lead inconsistent response (if you think, how you chat app is working)

Where you will put a load balancer in case of distributing the load to the WebSocket servers :

If you put your load balancer in L3 (Network layer) then your request will distribute according to your IP address.
service at the network layer will generate a hash of your IP address and send the request to the respective WebSocket server (Consistent hashing). network layer won’t maintain the state of the request

If you put your load balancer in L7(Application layer) then the load balancer has to maintain a state (which source IP-port pair is going to which backend node). which bad for resources.

I hope i clairify some of you doubt

I would suggest looking at some chat message system design architecture, HTTP (Keep-Alive) vs Websocket, How to pub/sub with WebSocket (it scale WebSocket very well)
This blog has good approach on how to scale WebSocket for multiple users: https://hackernoon.com/scaling-websockets-9a31497af051

Don’t miss nginix, its nice site for distributed system design

Leave a Comment