Round Robin Versus Least Connections
TradeoffNetworking & OSMidSeniorUpdated
Load balancing across backends where request durations vary wildly (some 1ms, some 5s). Round robin or least connections?
Short answer: Least connections. Round robin assumes equal-cost requests.
The Two Options
A
Round robin
B
Least connections
Drill This: Your Pick First
The Better Pick
Option B
Least connections
Round robin assumes equal-cost requests. With heavy-tailed durations it piles slow requests on whichever backend got unlucky. Least-connections routes new work to the least-loaded backend, smoothing tail latency.
When the Other One Wins
Round robin
For uniform, short, stateless requests with sticky-session needs, round robin (or consistent hashing) is cheaper and avoids the cross-LB connection-count bookkeeping. Least-connections also gets tricky across multiple LB instances.