Sweaty Imposter

Exponential Backoff + Jitter

TechniqueReliabilityMidSeniorUpdated
Grow retry intervals exponentially and add randomization so clients don't synchronize their retries.

When to Use Exponential Backoff + Jitter

Calling any unreliable downstream (network, third-party API, flaky DB). Without jitter, a thundering-herd of clients all retry at the same moment after an outage.

In Practice

AWS SDK clients retry with full jitter: sleep = random(0, base × 2^attempt). Both Google and Stripe SDKs follow the same pattern.

Sketch

Python

def retry(fn, max_attempts=5, base_ms=100, cap_ms=10_000):
    for attempt in range(max_attempts):
        try:
            return fn()
        except TransientError:
            if attempt == max_attempts - 1:
                raise
            backoff = min(cap_ms, base_ms * 2 ** attempt)
            jitter = random.uniform(0, backoff)
            time.sleep(jitter / 1000)

How Exponential Backoff + Jitter Comes Up in Interviews

Scenario

Your payments service comes back from a 5-minute outage and immediately gets knocked over again because every client's retry loop fires at the exact same instant. What's the standard fix?

Exponential backoff with full jitter randomizes retry timing so clients don't synchronize. Each waits a random interval up to base × 2^attempt, spreading load across time.

The 'thundering herd' problem is endemic to any large fleet of clients sharing a single dependency. After a brief outage, every client's retry timer fires at roughly the same instant, multiplying real load by N (the fleet size) for a few seconds, often knocking the recovering service straight back down. Exponential backoff alone helps but doesn't fully fix it because all clients still retry at the same intervals; you need jitter (randomization) on top, so the retry curve spreads across time instead of stacking. AWS SDKs, Stripe's client libraries, and gRPC's built-in retry policy all use 'full jitter' (sleep = random(0, base × 2^attempt)) as the default. Hedged requests and circuit breakers attack different problems (tail latency, cascading failure), so they're not substitutes here.

Scenario

Stripe sends you a webhook, your handler 500s, Stripe retries immediately. To prevent your handler from being hammered while you recover, Stripe spaces its retries at exponentially increasing intervals with randomization. What's the technique?

Exponential backoff with jitter is the standard for spacing retries against a struggling endpoint. A leaky bucket would smooth steady-state delivery but isn't what governs retry spacing after failures.

Stripe's webhook retry policy spaces attempts at exponentially increasing intervals (minutes, then hours, then days, with jitter) over up to 3 days, so a temporarily-down receiver isn't hammered with retries during recovery. The pattern is identical to AWS SDK and gRPC client retries internally: backoff prevents synchronized retry storms across a fleet of senders, jitter prevents synchronized retries from a single sender. The contrast with leaky bucket is sometimes confused: leaky bucket smooths steady-state outbound traffic to a fixed rate, while exponential backoff governs spacing of retries against a struggling endpoint. The two are complementary, Stripe likely uses both in different parts of the delivery pipeline.