Thread-per-Request With a Large Pool Versus Async / Event-Loop With Futures
TradeoffConcurrencyMidSeniorUpdated
A service makes ~5 downstream HTTP calls per request, 80% of request time is waiting on I/O, and you expect 50k concurrent connections.
Short answer: Async / event-loop with futures. I/O-bound workloads with high concurrency don't need a thread per in-flight call, the threads just sleep on epoll.
The Two Options
A
Thread-per-request with a large pool
B
Async / event-loop with futures
Drill This: Your Pick First
The Better Pick
Option B
Async / event-loop with futures
I/O-bound workloads with high concurrency don't need a thread per in-flight call, the threads just sleep on epoll. An event loop drives tens of thousands of concurrent I/Os with a handful of OS threads, cutting memory and context-switch overhead.
When the Other One Wins
Thread-per-request with a large pool
For CPU-bound work, threads (one per core) win, async buys nothing when nobody is sleeping.