Exponential Backoff and Retry Strategy

Naive retries do not just fail to help — they actively cause the outages they are trying to recover from. This article unpacks the contention model that makes exponential backoff with jitter the canonical answer, derives the three jitter algorithms from the 2015 AWS analysis, shows where retry budgets and circuit breakers fit, and walks through real postmortems where one or more of these defences was missing.

Mental model

Retry strategy is a contention-resolution problem. Multiple uncoordinated clients are competing for a recovering resource, and the goal is to spread their attempts out enough that the resource can drain its backlog before the next wave hits.

Three orthogonal mechanisms cooperate:

1. Backoff spreads load over time. The formula delay = min(cap, base × 2^attempt) creates exponentially growing inter-arrival windows: with base = 100 ms, the fifth retry waits up to 3.2 s, giving the server room to drain its queue.
2. Jitter spreads load across clients. Without randomness, a thousand clients that all failed at t = 0 will all retry at exactly t = 100 ms, then at t = 200 ms, then t = 400 ms — synchronised spikes that look like the original failure. Full jitter (random(0, calculated_delay)) replaces every spike with a near-uniform stream.
3. Budgets and breakers stop retries from amplifying further. A per-client retry budget caps how many of your outbound requests are retries; a circuit breaker cuts the call entirely when failures persist.

Why naive retries fail: contention resolution

The most damaging retry anti-pattern is the immediate retry. A while (true) with a fixed delay, replicated across thousands of callers, creates a feedback loop the industry has named the thundering herd: a recovering service is hit by a synchronised flood of retries that exhausts its connection pool, threads, or memory, and crashes it again before it can stabilise.

The same problem appeared in 1973 in the Aloha network and was carried into Ethernet: when multiple stations try to use a shared medium at once, mutual interference drops throughput to zero. The truncated Binary Exponential Backoff algorithm in IEEE 802.3 (CSMA/CD) is the same idea: after each collision, double the random window the station picks its next slot from, capped at 2^10 = 1024 slots, and give up after 16 attempts.¹

Service-to-service retries are the same problem at a different scale. The shared resource is a queue, a database, or a connection pool instead of a cable, but the contention dynamics — and the solution — are identical.

The mechanics of exponential backoff

Why exponential and not linear

Exponential growth matches the typical recovery curve of an overloaded system. When a queue is backed up, the time to drain it is proportional to the depth of the backlog, which itself grows with the duration of the incident. Linear backoff under-corrects; constant backoff does not correct at all.

The capped formula and its parameters

The canonical implementation is capped exponential backoff. The cap prevents delays from drifting into useless territory — a 30-minute retry serves nobody.

Contention math: why doubling the window is enough

For N independent clients picking uniformly random slots in a window of size W = 2^c after c collisions:

• The probability that any two specific clients pick the same slot is .
• The expected number of pairwise collisions is .

So the window has to grow with , not , to keep collisions rare. Doubling per collision (multiplying by 2 each time) catches up to a quadratic growth target in rounds, which is why the BEB doubling rule works in practice. With and (), the expected pairwise collision count is still around 488 — the network is still heavily contended at a single window, which is exactly why CSMA/CD also relies on transmission-level back-pressure beyond the backoff window.

The takeaway for service retries is that doubling alone does not magically eliminate contention; it makes contention recoverable. Jitter does the rest.

Jitter: decorrelating clients

The “thundering herd” in service retries

Pure exponential backoff is deterministic. A thousand clients that fail at the same instant will all calculate base × 2^1, all sleep for the same number of milliseconds, and all hit the recovering service at the same moment again. The shape of the load is unchanged; only the timing has shifted.

Jitter — controlled randomness on top of the calculated delay — breaks this correlation. Each client picks a different point in the backoff window, so a synchronised failure becomes a smooth, near-constant arrival rate at the server.

Three jitter algorithms (AWS, 2015)

The 2015 AWS Architecture Blog post by Marc Brooker is the canonical source for the three jitter strategies in production use today.

Full jitter (recommended default)

• Spread: maximum — clients scatter uniformly across the entire window.
• Trade-off: occasional near-zero delays, but the lowest server load of any approach in Brooker’s simulations. Decorrelated jitter finishes slightly faster on the same workload; full jitter trades a small amount of completion time for less work.
• In practice: this is the default in the AWS SDKs’ standard retry mode and the Google Cloud client libraries; pick it unless you specifically need a minimum spacing or want delays to grow off the previous draw.

Equal jitter

• Spread: half the window; guarantees a minimum wait of temp / 2.
• Trade-off: slightly more total work than full jitter, but bounded minimum delay.
• Use case: when the caller needs at least a known minimum to release a resource (close a connection, drain a buffer) before retrying.

Decorrelated jitter

• Spread: depends on the previous delay, not the attempt count.
• Trade-off: naturally adaptive — the window grows in response to actual conditions rather than a precomputed schedule.
• Caveat: once prev_sleep × 3 > cap, repeated clamping to cap causes consecutive delays to cluster near the cap, which re-correlates clients. Full jitter avoids this edge case entirely.

A production-ready implementation

A retry helper is small enough to write once and reuse everywhere. The four traits that distinguish a usable one from a toy:

• Configurability: every parameter is a knob.
• Pluggable jitter: the jitter function is an injected dependency.
• Error filtering: the helper knows the difference between transient and permanent errors.
• Cancellation: integrates with AbortSignal so callers can stop a retry chain mid-sleep.

Show 22 hidden lines23    initialDelay = 100,24    maxDelay = 30_000,25    backoffFactor = 2,26    jitterStrategy = fullJitter,27    isRetryableError = (error: unknown) => !(error instanceof Error && error.name === "AbortError"),28    abortSignal,29  } = options3031  let attempt = 032  let lastError: unknown3334  while (attempt <= maxRetries) {35    if (abortSignal?.aborted) {36      throw new DOMException("Aborted", "AbortError")37    }3839    try {40      return await operation()41    } catch (error) {42      lastError = error4344      if (!isRetryableError(error) || attempt === maxRetries || abortSignal?.aborted) {45        throw lastError46      }4748      const exponentialDelay = initialDelay * Math.pow(backoffFactor, attempt)49      const cappedDelay = Math.min(exponentialDelay, maxDelay)50      const jitteredDelay = jitterStrategy(cappedDelay)5152      console.warn(`Attempt ${attempt + 1} failed. Retrying in ${Math.round(jitteredDelay)}ms...`)5354      await delay(jitteredDelay, abortSignal)55      attempt++56    }57  }5859  throw lastError60}6162const delay = (ms: number, signal?: AbortSignal): Promise<void> => {Show 18 hidden lines

Show 22 hidden lines23async function resilientFetchExample() {24  const controller = new AbortController()2526  try {27    const data = await retryWithBackoff(() => fetchSomeData("https://api.example.com/data", controller.signal), {28      maxRetries: 4,29      initialDelay: 200,30      maxDelay: 5_000,31      jitterStrategy: equalJitter,32      isRetryableError: isHttpErrorRetryable,33      abortSignal: controller.signal,34    })35    console.log("Successfully fetched data:", data)36  } catch (error) {37    console.error("Operation failed after all retries:", error)38  }39}

The broader resilience ecosystem

Backoff with jitter is one layer in a stack. The other three layers it composes with — circuit breakers, retry budgets, and request hedging — each address a failure mode that backoff alone cannot.

Circuit breakers wrap retries; not the other way around

Backoff handles a single request that hit a transient error. A circuit breaker handles all requests to an endpoint that has been consistently failing. Wrapping retries inside a breaker means that once an endpoint has clearly broken, the breaker fast-fails new attempts without even invoking the retry loop.

The implementation order matters: the breaker must wrap the retry loop, not the other way around. The full sequence inside one call:

In a service mesh, both patterns can be configured at the infrastructure layer rather than in application code. Istio splits them across two CRDs:

1apiVersion: networking.istio.io/v12kind: VirtualService3spec:4  http:5    - retries:6        attempts: 37        perTryTimeout: 2s8        retryOn: 5xx,reset,connect-failure

1apiVersion: networking.istio.io/v12kind: DestinationRule3spec:4  trafficPolicy:5    outlierDetection:6      consecutive5xxErrors: 57      interval: 30s8      baseEjectionTime: 30s

Outlier detection ejects an upstream host once it has returned consecutive5xxErrors 5xx responses in a row, and keeps it ejected for baseEjectionTime × ejectionCount (the ejection time scales with how often a host has been kicked out). That is Envoy’s circuit-breaker primitive — and because the retry policy lives on a different resource, application code does not need to know about either.

Retry budgets cap retry amplification

Even with perfect jitter, retries amplify multiplicatively in deep call chains. A user request that fans out through three services, each retrying up to four times on failure, produces up to 4 × 4 × 4 = 64 attempts at the leaf — exactly when the leaf is least able to handle them.

Two complementary controls from the Google SRE book’s “Handling Overload” chapter bound the damage:

• Per-request limit: hard cap of three retry attempts per request. After that, propagate the error.

• Per-client retry budget: track the ratio of retries to total requests, and only retry when

  This caps worst-case amplification at roughly 1.1× per layer instead of 3× or worse. The same chapter also notes that retries should happen at the layer immediately above the rejecting one — never at every layer, because that is exactly what produces the multiplicative blow-up.

Backends help by signalling intent. gRPC distinguishes UNAVAILABLE (transient, OK to retry on the same or another backend) from RESOURCE_EXHAUSTED (you are being back-pressured — do not retry immediately). Google SRE’s pattern goes further: an overloaded backend can reply with an explicit “overloaded; don’t retry” status that tells the caller other backends in the cell are likely also overloaded, so retrying anywhere is futile.²

Honouring server signals: Retry-After and friends

When a server replies 429 Too Many Requests or 503 Service Unavailable, it can include a Retry-After header (RFC 9110 §10.2.3) telling the client how long to wait. The header takes one of two forms:

• Retry-After: 120 — seconds to wait (delay-seconds).
• Retry-After: Wed, 21 Oct 2026 07:28:00 GMT — absolute time (HTTP-date).

A correct client honours Retry-After when present, falls back to its own backoff calculation otherwise, and adds jitter even when honouring the header so that a thousand clients told to retry in 120 s do not all retry at exactly 120 s.

Show 5 hidden lines6  if (!Number.isNaN(seconds)) return seconds * 100078  const date = Date.parse(retryAfter)9  if (!Number.isNaN(date)) return Math.max(0, date - Date.now())1011  return calculatedDelay12}

The AWS SDK retry behavior docs describe the same idea at the SDK level. The default standard mode does jittered exponential backoff and circuit-breaks on persistent failure. The adaptive mode (still marked experimental at the time of writing) adds a client-side rate-limiter that maintains a token bucket and drops or delays even initial requests when throttling responses come back. Adaptive mode is powerful but assumes the client is scoped to a single resource — a single S3 bucket, one DynamoDB table — because throttling on one resource will throttle all traffic through that client.

Request hedging: a different problem entirely

Retries handle failures. Hedging handles tail latency — the small fraction of requests that take an order of magnitude longer than the median because of a GC pause, a cold cache, or a hot shard. The 2013 Tail at Scale paper showed that deferring the duplicate request until the primary has been outstanding past the 95th-percentile latency keeps the extra load to roughly 5 % while collapsing the tail.

gRPC’s hedging configuration sends parallel copies after hedgingDelay and cancels the others as soon as one succeeds:

1{2  "hedgingPolicy": {3    "maxAttempts": 3,4    "hedgingDelay": "0.5s",5    "nonFatalStatusCodes": ["UNAVAILABLE", "UNKNOWN"]6  }7}

Operationalising backoff

Idempotency is non-negotiable for write retries

For idempotent operations (GET, HEAD, PUT, DELETE), retries are safe by construction. For non-idempotent ones (POST, charge a card, dispatch an SMS), the API has to be explicitly designed to be idempotent — usually by accepting a client-generated key in an Idempotency-Key header that the server uses to deduplicate.

The semantics are pinned down in the active IETF draft draft-ietf-httpapi-idempotency-key-header (currently at -07, October 2025). Briefly:

• The client generates a unique key per logical operation and includes it in the Idempotency-Key request header.
• A first-time key triggers normal processing; the server caches the result against the key.
• A retry with the same key and same payload returns the cached result.
• A retry while the original is still in flight should return 409 Conflict.
• Reuse of the key with a different payload should return 422 Unprocessable Content.

Stripe, Square, AWS, and others have run variants of this pattern in production for years; the draft codifies what was already de facto practice.

Tuning parameters from production data

Library defaults are starting points, not answers. Tune from your own metrics:

• Per-attempt timeout: a common heuristic from the AWS Builders’ Library is to set it slightly above the downstream’s healthy p99 or p99.5 latency. Lower is too aggressive (you cancel real work); higher leaks slow tail latency back to the caller.
• maxRetries: derive from the operation’s user-facing latency budget and from the service SLO/error-budget. A user-facing path with a 2 s budget and a 200 ms p50 cannot afford five retries with a 30 s cap.
• base: at least one healthy round-trip (typically ≥ 100 ms); too low and the first retry hits before the server can drain.
• cap: a few times the typical incident duration; for most internal services 30 s is plenty.

Observability: what to log and dashboard

For every retry, log:

• The operation being retried.
• The attempt number (e.g. retry 2 of 5).
• The calculated delay before jitter, the final jittered delay, and the actual sleep.
• The error type and any retry-relevant headers (Retry-After, x-amzn-RequestId, gRPC status code).

For the system as a whole, dashboard:

• Retry rate per second per endpoint and per call site.
• Success rate after retries vs first-attempt success rate.
• Final failure rate after the budget is exhausted.
• Circuit breaker state (Closed / Open / Half-Open) and transition counts.
• Retry delay histogram to confirm jitter is actually spreading load.

When retry rate spikes without a matching error rate, your retry budget or breaker thresholds are wrong; when retries climb but never converge, the upstream is broken in a way that retries cannot fix.

Real-world failures

Three postmortems cover the failure modes the layered defence is designed to prevent.

Discord: thundering herd from a flapping service

Discord’s March 2017 connectivity incident (catalogued in Dan Luu’s postmortems list) is a textbook thundering herd. A presence-tracking server soft-locked and netsplit from its cluster; the millions of Erlang processes inside the sessions service all attempted to reconnect to the presence cluster at once, overloading the remaining presence nodes. While presence was unavailable, the sessions service’s per-connection buffers filled with un-deliverable events, causing one-third of session nodes to OOM. Bringing presence back triggered the same reconnection storm a second time.

Lesson: a service-to-service reconnection is a retry too, and it needs jittered backoff and a hard in-flight cap. The Discord post-incident actions were exactly that — a connection limit on the sessions→presence dependency and a fast-fail path so a presence outage stops cascading into sessions. Without those, every recovery synchronises into the next outage.

Google SRE: retry amplification across layers

The Google SRE book’s Cascading Failures chapter describes a deeply-nested architecture where a database started failing under load. Each of three layers above it implemented its own retry policy with up to four attempts, multiplying load by 4³ = 64× at exactly the moment the database was least able to handle it.

Lesson: retry at one well-chosen layer (usually the layer immediately above the one rejecting requests). The same chapter recommends retry budgets and the “overloaded; don’t retry” backend signal as the second line of defence.

Twilio: non-idempotency turns a config bug into a billing incident

The Twilio billing incident postmortem from July 2013 shows what happens when retry logic meets a non-idempotent operation. The proximate cause was unrelated to retries: when engineers restarted a Redis master after a network partition, it loaded the wrong config file and came up as a slave-of-itself in read-only mode. All balance writes silently failed.

The amplifying cause was that the auto-recharge code was not idempotent: every usage event (an SMS, a call) checked the account balance, saw zero, and re-attempted to charge the customer’s credit card. Because the balance write was failing, every subsequent event re-triggered the charge. Roughly 1.4% of customers were billed multiple times for the same operation.

Lesson: idempotency is a property of the operation, not of the retry helper. A retry library cannot make a charge-card-then-update-balance flow safe. Either the operation has to be redesigned around an idempotency key, or it must not be subject to automated retry at all.

Practical takeaways

A retry strategy is a system property, not a function call. The defaults that hold up under load:

1. Default to capped exponential backoff with full jitter (delay = min(cap, base × 2^attempt), sleep = Uniform(0, delay)). Use base ≈ 100 ms, factor = 2, cap = 30 s, maxAttempts = 3-5 and tune from there.
2. Wrap retries inside a circuit breaker. Backoff handles transient errors; the breaker handles sustained failure. They compose; they don’t substitute for each other.
3. Cap retry amplification with a per-client retry budget (~10% retry-to-request ratio) and retry at one layer only.
4. Honour Retry-After when present, and add jitter on top of it.
5. Make writes idempotent before you retry them. Idempotency-Key is the standard pattern; build it in early.
6. Instrument the retry path so you can see whether retries are recovering from real transient errors or amplifying a permanent one.

The discipline is not “retry harder when things fail.” It is “retry only when retrying is likely to help, only as much as your budget allows, only at the right layer, and only for operations that are safe to repeat.”

Appendix

Prerequisites

• HTTP status code semantics (4xx vs 5xx, 429, 503).
• async/await and AbortSignal in JavaScript/TypeScript.
• Distributed-systems vocabulary: latency, availability, percentile, idempotency.

Terminology

References

Standards and specifications

• RFC 9110 - HTTP Semantics, §10.2.3 Retry-After
• draft-ietf-httpapi-idempotency-key-header-07 — Idempotency-Key HTTP header (active IETF WG draft, October 2025)
• IEEE 802.3 — CSMA/CD with truncated BEB
• gRFC A6 — gRPC Client Retries (and Hedging)

First-party documentation

• AWS Architecture Blog — Exponential Backoff and Jitter (2015) — canonical jitter analysis
• AWS Builders’ Library — Timeouts, Retries, and Backoff with Jitter
• AWS SDK retry behavior — standard vs adaptive modes
• Google SRE Book — Handling Overload — retry budgets and “overloaded; don’t retry”
• Google SRE Book — Addressing Cascading Failures — retry amplification math
• Google Cloud Storage — Retry Strategy
• gRPC — Retry guide and Request Hedging
• Istio — Circuit Breaking and DestinationRule reference
• Azure — Retry Pattern and Circuit Breaker Pattern

Postmortems

• Twilio — Billing Incident Post-Mortem (2013)
• Discord — March 2017 connectivity incident (dj3l6lw926kl, thundering herd) (via danluu/post-mortems)

Practitioner write-ups and reference implementations

• Marc Brooker — Jitter: Making Things Better With Randomness
• Marc Brooker — What is Backoff For?
• aws-samples/aws-arch-backoff-simulator — simulator for the 2015 jitter analysis
• Code samples and tests for the retry helper above