Sweaty Imposter

Hedged Requests

TechniqueReliabilitySeniorStaffUpdated
If the first request hasn't returned by some threshold (p95, the 95th percentile latency), fire a second one and take whichever finishes first.

When to Use Hedged Requests

Tail-latency-sensitive reads where the long tail is wasted (e.g., a single slow replica). Costs ~2x reads on the tail but slashes p99 (99th percentile) latency.

In Practice

Google's BigTable + Spanner read clients hedge to a second replica after ~95th percentile of recent latency. Used internally by search to keep query latency consistent.

How Hedged Requests Comes Up in Interviews

Scenario

Your read path hits a 3-replica datastore. p50 is fine but p99 (99th percentile latency) is dominated by occasional slow replicas. You want to cut tail latency and can afford ~2x reads on the tail. What do you reach for?

Hedged requests fire a second request to another replica once the first crosses a p95 (95th percentile latency) threshold and take whichever returns first, directly attacking tail latency. Quorum reads address consistency, not tail latency.

Hedged requests come straight out of Jeff Dean's 'Tail at Scale' paper, which observed that even with 1% slow responses per server, a request that fans out to 100 servers has a 63% chance of being slow somewhere. The hedge fires a duplicate to a different replica once latency crosses a threshold (typically the recent p95) and takes whichever returns first, paying maybe 5-10% extra reads to cut p99 by 30-50%. Google's internal BigTable and Spanner clients hedge by default; so does TiKV. The tradeoff is that hedging amplifies load on the slow replica too: if the slowness is from overload rather than transient weirdness, hedging makes the cascade worse, which is why you usually combine it with a circuit breaker.

Scenario

Google's internal BigTable client kicks off a second read to a different replica if the first hasn't returned within roughly the p95 (95th percentile latency) of recent latencies, then takes whichever finishes first. What's this technique called?

This is the textbook 'hedged requests' pattern from Dean's Tail at Scale paper, trading a small amount of extra read traffic on the tail for a large p99 (99th percentile latency) win.

Hedged requests come directly from Jeff Dean's 'Tail at Scale' paper, where Google reported that hedging at the p95 threshold cuts BigTable p99 by ~40% at a cost of only ~5% extra reads. The idea generalizes: any time the long tail of latency is dominated by occasional slow servers (GC pauses, transient network hiccups, hot replicas), firing a duplicate request and taking whichever returns first is dramatically cheaper than letting the tail dominate user-perceived latency. Spanner, MapReduce, and many internal Google systems hedge by default. The technique is now built into many service mesh libraries (Linkerd, Envoy) as a config option.