Stripe: Idempotency for Payment Reliability

Stripe handles network ambiguity in a payment API the way a database handles a power loss: it makes every state transition recoverable, then retries to convergence. The mechanism is a small set of patterns — Idempotency-Key requests, atomic phases bracketing every external call, recovery points that survive process death, and jobs staged inside the same transaction that produced them. This article walks through how those pieces fit together at the scale Stripe operates ($1 trillion in 2023 payment volume on a 99.999%-uptime document store)¹², what changed when API v2 shipped, and how to apply the same model to your own system.

Why payment APIs need exactly-once semantics

A network call can fail in three ways that look identical from the client side:

1. The connection never reaches the server.
2. The server accepts the request, mutates state, then crashes before responding.
3. The server completes the request, but the response is lost in transit.

Only the first is unambiguously safe to retry. The other two leave the client guessing whether a charge already happened. For a card-present flow this is recoverable through human reconciliation; for a programmatic API it shows up as duplicate charges, broken inventory, and chargebacks.

Stripe’s response is not a more clever retry algorithm — it is a contract between client and server: the client picks a unique key per logical operation and includes it on every retry; the server promises that the same key produces the same observable outcome³. The client no longer has to know whether the server saw the original request.

Mental model: the database as the coordinator

The pattern Stripe popularised — and that Brandur Leach documented in detail when he was at Stripe — sits on three load-bearing ideas³:

The single most important rule: commit local state before initiating any foreign-state mutation. Once the request has crossed your boundary (Stripe API, mailer, Kafka, another microservice), you can’t roll it back; you can only record that it happened and either retry it idempotently or compensate.

The idempotency-key row

Stripe stores idempotency keys per account in a row that tracks the request through its lifecycle. The Rocket Rides reference implementation⁶ uses Postgres; Stripe internally uses DocDB (Stripe’s MongoDB-derived document store)², but the schema is functionally equivalent.

1CREATE TABLE idempotency_keys (2    id              BIGSERIAL   PRIMARY KEY,3    idempotency_key TEXT        NOT NULL,4    user_id         BIGINT      NOT NULL,5    locked_at       TIMESTAMPTZ,6    request_method  TEXT        NOT NULL,7    request_path    TEXT        NOT NULL,8    request_params  JSONB       NOT NULL,9    response_code   INT         NULL,10    response_body   JSONB       NULL,11    recovery_point  TEXT        NOT NULL DEFAULT 'started',12    last_run_at     TIMESTAMPTZ NOT NULL DEFAULT now(),13    created_at      TIMESTAMPTZ NOT NULL DEFAULT now(),14    CONSTRAINT idempotency_keys_user_key_unique15        UNIQUE (user_id, idempotency_key)16);

Four design decisions shape every interaction with this table:

• Per-account scope. The unique key is (user_id, idempotency_key), not the key alone. Two unrelated accounts can pick the same UUID without colliding³.
• locked_at as a soft lock. The first request to insert the row also locks it; concurrent requests for the same key see the lock and return 409 Conflict. The lock is cleared when the row reaches finished or when an error path explicitly releases it.
• recovery_point as a state-machine cursor. Each atomic phase ends by writing the next recovery point in the same transaction it commits work into. A retry reads the cursor and jumps to the matching phase.
• request_params for fingerprinting. Reusing the same key with a different body is almost always a bug, and Stripe rejects it with 409 Conflict and a idempotency_error type⁴⁷.

Key requirements (Stripe-specific)

Atomic phases: bracketing every foreign call

An atomic phase is a SERIALIZABLE Postgres transaction whose body executes only local mutations. The boundary of a phase is the next foreign-state mutation — the call to Stripe, the call to the mailer, the publish to Kafka. Brandur’s three rules for finding phase boundaries³:

1. The idempotency-key upsert is its own phase.
2. Every foreign-state mutation is its own phase.
3. Every other operation between two foreign mutations gets grouped into one phase, regardless of how many DB rows it touches.

The Rocket Rides flow has one foreign call (the Stripe charge) and produces three local atomic phases plus the foreign mutation between them.

Why SERIALIZABLE

Stripe (and the Rocket Rides implementation) uses the strictest transaction isolation level available in Postgres. The reason is the idempotency-key upsert: two requests with the same key arriving within microseconds need a guarantee that exactly one of them creates the row and the other gets a 409 Conflict. Lower isolation levels can permit the write-skew anomaly that breaks this invariant³.

The cost is that concurrent transactions touching the same row pair can abort with Sequel::SerializationFailure, surfacing to the client as 409 Conflict. The official guidance is to retry on the client; the SDKs already do this⁷.

1def atomic_phase(key, &block)2  DB.transaction(isolation: :serializable) do3    ret = block.call4    case ret5    when RecoveryPoint6      key.update(recovery_point: ret.name)7    when Response8      key.update(9        recovery_point: 'finished', locked_at: nil,10        response_code: ret.status, response_body: ret.body11      )12    when NoOp13      # phase touched nothing on the key14    end15  end16rescue Sequel::SerializationFailure17  halt 409, { error: 'concurrent request' }.to_json18end

Each phase returns one of three sentinel values: a new RecoveryPoint, a terminal Response (success or unrecoverable error), or a NoOp if the phase was a probe (e.g. the upsert of an already-finished key)³.

Recovery points are a DAG, not a loop

Recovery points form a directed acyclic graph anchored at started and absorbed at finished. Every successful phase advances the cursor; every unrecoverable error short-circuits straight to finished with the response cached on the row.

The driver loop is a case over recovery_point that runs the matching phase and lets the loop iterate again with the new cursor value. A request that starts at charge_created because a previous attempt crashed mid-flow will skip directly to phase 3³.

Foreign-state mutations: the constraint that drives the design

Once you call out to Stripe, send an email, or publish to a downstream system, you cannot atomically undo it. That single fact is what forces the atomic-phase pattern: you must commit a local record of intent to call before the call, and a local record of what happened after, so that retries can be reasoned about.

When the foreign API also speaks Idempotency-Key, you derive its key deterministically from your own row’s primary key. The Rocket Rides example uses rocket-rides-atomic-#{key.id}³:

1charge = Stripe::Charge.create(2  {3    amount: 20_00, currency: 'usd',4    customer: user.stripe_customer_id,5    description: "Charge for ride #{ride.id}",6  },7  {8    idempotency_key: "rocket-rides-atomic-#{key.id}",9  },10)

Now both your own retries and Stripe’s internal retries are safe: every attempt sends the same derived key to Stripe, and Stripe’s idempotency layer guarantees the charge is created exactly once⁴.

When the foreign API isn’t idempotent

Brandur’s framing is the right one: if a foreign API doesn’t support idempotency keys and you see an indeterminate failure (timeout, connection reset), you have to mark that operation as permanently failed. You cannot safely retry, because you don’t know whether the side effect already happened³. That is why Stripe pushes idempotency support so hard onto its own ecosystem — every adopter raises the floor for everyone else.

Mapping foreign errors to phase outcomes

Transactionally-staged job drains

The natural temptation when writing the receipt-email step is to enqueue a Sidekiq job from inside the transaction:

1DB.transaction do2  ride = Ride.create(...)3  Sidekiq.enqueue(SendReceiptJob, ride.id)   # WRONG4end

Two failure modes here, both subtle and both production-real⁹:

1. Worker beats the commit. Redis is fast. The worker dequeues, queries Postgres for ride.id, and finds nothing — the outer transaction hasn’t committed yet.
2. Crash between commit and enqueue. Move the enqueue after the commit and the bug inverts: the row is durable, but the process dies before Redis sees the job. The receipt is silently lost.

The transactionally-staged-job-drain pattern fixes both by making the job insert part of the same transaction as the work it follows up on, then having a separate enqueuer process drain the staging table to the real queue⁹.

1CREATE TABLE staged_jobs (2    id       BIGSERIAL PRIMARY KEY,3    job_name TEXT      NOT NULL,4    job_args JSONB     NOT NULL5);

1acquire_lock(:enqueuer) do2  loop do3    DB.transaction(isolation_level: :repeatable_read) do4      jobs = StagedJob.order(:id).limit(BATCH_SIZE)5      next if jobs.empty?6      jobs.each { |j| Sidekiq.enqueue(j.job_name, *j.job_args) }7      StagedJob.where(Sequel.lit('id <= ?', jobs.last.id)).delete8    end9    sleep_with_exponential_backoff10  end11end

The properties this gives you are stronger than they look:

• No lost jobs. Rows only leave the staging table after Redis acknowledges the enqueue, and the enqueuer deletes them inside its own transaction.
• No phantom jobs. A rolled-back outer transaction takes the staged row with it.
• At-least-once delivery, naturally. A crash between enqueue and delete causes a re-enqueue on restart; downstream workers must already be idempotent (and yours will be, because you’ve been reading this article).

Background recovery: the completer

Two things can leave a key stuck mid-flow: the originating client gives up before retrying, or a process crash leaves locked_at set with no live request behind it. The completer is a background process that picks up these orphans and pushes them through to finished. The Rocket Rides implementation runs in a loop, querying for keys whose locked_at is older than a grace period (5 minutes in the reference, longer in practice), and replays them via the same atomic-phase pipeline³.

1GRACE = 5 * 60 # seconds23loop do4  IdempotencyKey5    .where('recovery_point != ?', 'finished')6    .where('locked_at < ?', Time.now - GRACE)7    .each { |key| resume_from_recovery_point(key) }89  sleep 110end

The grace window has to outlast a slow-but-active request. Too short and the completer races a still-live request, double-locking; too long and abandoned work piles up. Stripe doesn’t publish its production value; the reference uses 5 minutes, and that’s a sane default unless your phases are unusually long³.

Reaping: the row isn’t permanent

Idempotency keys are bounded-storage primitives. They exist to make the near-term retry safe, not to act as an audit log. A reaper deletes rows that have outlived the retention horizon.

After expiry, a retry with the same key is treated as a brand-new request⁴. Defences against the resulting duplicate-operation risk:

• Pick natural idempotency where possible (PUT over POST, “update or create” over “create”).
• Add a business-level dedup check (does a charge for this order_id already exist?).
• Move to API v2’s 30-day window for workflows that span days.

What changed in API v2

API v2 is a parallel namespace under /v2 that ships the post-2024 design patterns. Idempotency is one of the surfaces that got the most attention⁵.

Source: API v2 overview — Idempotency differences between API v1 and API v2⁵.

The v2 retry behaviour is the most significant practical change. In v1, a transient infrastructure failure that produced a 500 was permanently cached: every subsequent retry of that key got the same 500 back, and the client had to mint a fresh key to actually try again⁴⁷. In v2, the server attempts to re-execute the failed request without producing extraneous side effects and returns the new outcome⁵. For payment flows that span asynchronous webhooks and human action, the longer 30-day window lets the client safely retry a key it generated days earlier.

Error handling at the HTTP layer

Stripe’s HTTP status code reference⁷:

The Stripe-Should-Retry response header is the authoritative signal — true says retrying is safe, false says don’t bother, missing means the API can’t determine. Stripe SDKs honour it automatically⁷.

The 409 paths

Two distinct conditions surface as 409 Conflict with type: "idempotency_error":

• Concurrent in-flight request with the same key. The lock on the row is held; the second request loses⁴.
• Parameter mismatch — same key, different request body. Stripe returns the well-known message “Keys for idempotent requests can only be used with the same parameters they were first used with”⁷¹⁰.

Both are recoverable, but only one calls for waiting and retrying — the other calls for a new key.

Retry strategy on the client

Idempotency makes retries safe; it doesn’t make them free. Without bounded backoff and jitter, every recovering server immediately gets crushed by the thundering herd of clients that synchronised their retry schedules during the outage¹¹.

The standard recipe — already implemented in the Stripe SDKs — is exponential backoff with full jitter:

[ \text{sleep} = \min\bigl(\text{base} \cdot 2^{n-1},\ \text{cap}\bigr); \quad \text{actual} = \text{rand}(0,\ \text{sleep}) ]

The Stripe Ruby SDK ships the capability disabled by default; configure it explicitly¹²:

1Stripe.max_network_retries = 2

Once enabled, the SDK auto-generates idempotency keys for POST requests that don’t supply one, retries network errors and 429 lock timeouts on an exponential-backoff schedule with jitter, and respects the Stripe-Should-Retry header⁷¹¹.

Rate limits and how they interact with idempotency

The basic global limits¹³:

Per-resource limits are layered on top (Payment Intents has 1000 updates/hour per intent, the Files API caps at 20 reads + 20 writes/sec, etc.). When a request is rate-limited, Stripe returns 429 with Stripe-Rate-Limited-Reason set to one of global-rate, global-concurrency, endpoint-rate, endpoint-concurrency, or resource-specific¹³. Crucially, the rate limiter runs before the idempotency layer, so the same key remains safe to use on the retry — the request never reached the idempotency machinery in the first place⁷.

Object lock timeouts surface the same way (429 lock_timeout); they have a different cause (concurrent mutations on the same Stripe object) but the same mitigation — retry on the same key with backoff¹³.

Webhooks: a separate at-least-once channel

Idempotency keys handle the outbound request from your client into Stripe. Webhooks handle the inbound event from Stripe into your endpoint — and they have their own delivery semantics:

• At-least-once delivery. Stripe retries deliveries that don’t return 2xx for up to three days in live mode (a few hours in sandbox)¹⁴.
• No ordering guarantee. Two events in the same logical sequence may arrive out of order; design your handler to be commutative.
• Duplicate events possible. Even a single delivery can occasionally produce two Event objects with the same data.object ID and event.type; dedupe on (event.id, event.type, data.object.id)¹⁴.
• Signed for authenticity. Each delivery includes a Stripe-Signature header carrying a timestamp and one or more HMAC-SHA256 signatures over {timestamp}.{raw_body}. Stripe’s libraries default to a 5-minute tolerance window to mitigate replay attacks¹⁴.

A correct handler is short:

1post '/webhooks/stripe' do2  payload = request.body.read3  sig     = request.env['HTTP_STRIPE_SIGNATURE']4  event   = Stripe::Webhook.construct_event(payload, sig, ENDPOINT_SECRET)56  return 200 if ProcessedEvent.exists?(event_id: event.id)78  DB.transaction do9    ProcessedEvent.create(event_id: event.id)10    handle(event)11  end12  20013end

The signature verification rejects forged events; the ProcessedEvent insert (with a unique constraint on event_id) makes duplicate deliveries idempotent¹⁴. Combined with a transactionally-staged-job-drain for any heavy follow-up work, you can return 200 quickly and let downstream processing happen at its own pace.

Trade-offs and limits

The pattern optimises for correctness over throughput in three measurable ways:

This pattern doesn’t fit cleanly when:

• The foreign API isn’t idempotent. You can still record intent, but you can’t safely auto-retry indeterminate failures.
• The data store doesn’t offer real ACID transactions. Atomic phases collapse — every operation becomes a foreign-state mutation³.
• The work is fan-in/fan-out across regions where serialisable isolation has unbounded coordination cost. Look at sagas with explicit compensations instead.
• Throughput dwarfs correctness needs (analytics ingest, telemetry). Use simpler at-least-once with downstream dedup.

IETF standardisation

The pattern Stripe popularised is on its way to becoming a formal HTTP header. draft-ietf-httpapi-idempotency-key-header-07 (last revised 2025-10-15, expires 2026-04-18 in its current form)¹⁵ specifies an Idempotency-Key request header with structured-field syntax, prescribed semantics for retries vs. concurrent requests, and recommended error responses (409 for in-flight, 422 for fingerprint mismatch). The draft’s “Implementation Status” appendix lists the surface area of adoption¹⁵:

• Idempotency-Key header (matches Stripe’s spelling): Stripe, Adyen, Dwolla, Interledger, WorldPay, Yandex Cloud, Finastra, Datatrans, http4s.
• Different header name, same concept: PayPal (PayPal-Request-Id, 45-day retention)¹⁶, Razorpay (X-Payout-Idempotency), Twilio (I-Twilio-Idempotency-Token for webhooks), Chargebee (chargebee-idempotency-key), Open Banking UK (x-idempotency-key), BBVA (X-Unique-Transaction-ID).
• Body field instead of header: Square (idempotency_key in body), Google Standard Payments (requestId in body).

The draft’s IESG state currently shows “Expired”, but it’s been advancing through revisions for several years and the latest revision is recent¹⁵. Until it ships as an RFC, Stripe’s Idempotency-Key is the de facto standard most adopters mirror.

Applying this to your own system

If your system has any of these properties, the pattern earns its complexity:

• A non-idempotent operation whose accidental repetition has business cost (charges, inventory decrements, notifications, account creation).
• An ACID-compliant primary store (Postgres, MySQL, CockroachDB, Spanner, well-configured MongoDB).
• A foreign-state mutation (third-party API, transactional email, message broker) that supports its own idempotency mechanism.

The minimal version is genuinely small: the table, an upsert with SERIALIZABLE isolation, response caching, parameter fingerprinting, and a 4xx/409 on mismatch. That covers the bulk of real production failures. Add recovery points and the completer when you have multiple atomic phases or long-running workflows; add the staged-job drain when you start emitting follow-up work.

The reference implementation lives at brandur/rocket-rides-atomic — about 700 lines of Ruby, including tests, that exercise every code path described above³.

References