Synchronous I/O With a Large Thread Pool Versus Asynchronous Non-Blocking I/O With Futures/Await
TradeoffConcurrencyMidSeniorUpdated
Your service issues many DB and HTTP calls per request. You're choosing the I/O style for the platform.
Short answer: Asynchronous non-blocking I/O with futures/await. Non-blocking I/O scales connection count and concurrency far past what threads can, Netty / Tokio / asyncio routinely drive 100k connections per box.
The Two Options
A
Synchronous I/O with a large thread pool
B
Asynchronous non-blocking I/O with futures/await
Drill This: Your Pick First
The Better Pick
Option B
Asynchronous non-blocking I/O with futures/await
Non-blocking I/O scales connection count and concurrency far past what threads can, Netty / Tokio / asyncio routinely drive 100k connections per box. Threads cost ~1MB of stack each and context-switch overhead grows with the pool.
When the Other One Wins
Synchronous I/O with a large thread pool
Simple internal tools or batch jobs with low concurrency, sync code is dramatically easier to reason about and debug. The cognitive tax of async isn't worth it under low load.