Webhook Retries and Idempotency: Building Reliable Delivery Into Your Integration Layer

Every webhook provider makes the same quiet promise: "we'll deliver this event." Almost none of them promise to deliver it exactly once. Stripe retries failed deliveries for about three days. Shopify tries roughly nineteen times over 48 hours before removing the endpoint. GitHub, notoriously, does not retry failed deliveries at all — if your endpoint was down, the event is simply gone unless you go fetch it by hand.

That inconsistency is the real engineering problem behind webhooks. It isn't "how do I receive an HTTP POST" — that part is trivial. It's "what happens when the network lies to both sides at once," and what you build to survive that.

This is a practical guide to the two things that make webhook-based integrations trustworthy: retry behavior on the sending side, and idempotent handling on the receiving side. If you're building a product that depends on webhooks from Stripe, HubSpot, Shopify, or your own customers' systems — or you're the one sending webhooks out to your customers — this is the part of the architecture that determines whether a bad five minutes turns into a support ticket or a double charge.

HTTP gives you none of the guarantees a reliable event system needs. A POST either gets a response or it doesn't, and when it doesn't, neither side can be sure what happened on the other end.

Consider the failure that causes the most damage in production: your handler receives the webhook, processes it fully — charges a card, provisions an account, sends a confirmation email — and then the response gets lost before the provider sees your 200. From the provider's point of view, this looks identical to a dropped request. It has no way to distinguish "never arrived" from "arrived, processed, and the acknowledgment got lost." So it does the only safe thing: it retries.

That single ambiguity is why at-least-once delivery is the default guarantee almost every serious webhook provider ships, and why duplicates aren't a bug in their system — they're the price of not silently losing events. True exactly-once delivery would require the provider to know your internal processing state, which no stateless HTTP exchange can provide. What some platforms market as "exactly-once" is really at-least-once delivery plus a stable event ID and a recommendation that you deduplicate.

A webhook delivery is considered failed under a few conditions, and they're worth being precise about because they tell you what the retry logic is actually protecting against:

• Timeouts — your handler does slow synchronous work (an external API call, a database transaction) and blows past the provider's response deadline, often just a few seconds.

• Transient infrastructure errors — a 502/503 during a deploy, a database connection pool briefly exhausted, a pod being rescheduled mid-request.

• Network failures — the connection drops after you've processed the event but before the 200 reaches the sender.

• Non-2xx responses — anything other than a 2xx status, including an uncaught exception that surfaces as a 500, tells the provider "try again."

Providers that take this seriously implement exponential backoff with jitter: each retry waits longer than the last (seconds, then minutes, then hours), and a small random offset is added so a fleet of simultaneously-failing deliveries doesn't retry in lockstep and create a thundering herd against a server that's already struggling. The retry window varies a lot by provider — Stripe spreads retries over roughly three days, Shopify disables an endpoint after about 19 hours of continuous failure, Slack gives you three attempts over a few minutes, and GitHub, again, gives you none. If you integrate with more than one of these, you cannot assume a uniform retry contract; you have to look each one up, because "the provider will retry" is not a safe default without checking.

When retries exhaust, a well-built sender routes the failed delivery into a dead-letter queue rather than discarding it — a place for a human or an automated job to inspect and replay later. If you're the one sending webhooks to your own customers (for example, notifying a connected product when a workflow completes), building a DLQ and a replay path is not optional polish; it's the difference between a transient customer-side outage costing you nothing and costing you a support escalation.

Retries guarantee the sender eventually gets acknowledgment. They also guarantee you will, at some point, receive the same event more than once. If your handler isn't built for that, a duplicate delivery means a double charge, a duplicate shipment, or a customer getting the same "your order shipped" email three times.

Idempotency means processing the same event once, twice, or a hundred times produces the same end state as processing it once. The pattern is short and well understood, but the details matter:

1. Use the provider's event ID, not your own. Stripe sends id (evt_...), GitHub sends X-GitHub-Delivery, Standard Webhooks senders send a webhook-id header. A retry of the same delivery reuses this ID; a genuinely new event never does. Don't hash the payload or use a timestamp — both can legitimately differ across what is semantically the same delivery.

2. Record the ID before or atomically with the side effect, not as a separate check-then-act step. If you SELECT to check for existence and then INSERT to record it, two concurrent retries can both read "not present" and both proceed — a race, not an edge case, under real retry traffic. Use a unique constraint on the event ID column and let the database reject the duplicate insert, or wrap the check and the write in one transaction.

3. Respond 200 on a duplicate, not an error. When the insert violates the unique constraint, return success immediately. Returning a 409 or 500 tells the provider to retry again, and the retry hits the same constraint — consuming your retry window without resolving anything.

4. Extend idempotency past your own boundary. An event ID check stops you from processing the same event twice. It does not stop you from making a duplicate outbound call if the webhook handler itself calls another API — charging a card, creating a refund. Pass your own idempotency key, derived from the event ID, to that downstream call. Most payment APIs (Stripe, Adyen) support this natively and will return the original result instead of performing the action again.

5. Don't trust delivery order. Retries and parallel delivery workers mean a retried older event can arrive after a newer one. A handler that blindly applies whatever arrives last can reactivate a subscription that a later cancellation event already closed. Compare a timestamp or version field inside the payload against what you've already stored, and discard anything stale — or, for state that must be correct, treat the webhook purely as a "something changed, go check" signal and re-fetch the canonical object from the source API rather than trusting the payload as data.

In production, the handler that survives real traffic looks less like "receive event, do the work" and more like a two-stage pipeline:

Accept fast, process later. Verify the request, record the raw payload and event ID the moment it arrives, and return 200 immediately — before doing any slow work. This decouples the provider's retry window (which only has to cover whether you accepted the event) from your own processing time, and lets you own your own retry and dead-letter logic downstream instead of being at the mercy of the sender's schedule.

Reconcile on a schedule, not just on webhook arrival. Webhooks are a real-time notification mechanism, not a database. A provider outage, a window where your endpoint didn't exist yet, or a silently dropped delivery (GitHub's no-retry policy, for instance) will all create gaps that no amount of retry logic on the sender's side can close. A periodic job that polls the provider's API for anything changed since your last successful sync is the only backstop that catches what webhooks miss entirely.

Webhook reliability doesn't exist in isolation — it's one layer of a larger set of problems that show up the moment a product has to handle integration traffic for more than one customer. The same deduplication and backoff logic has to hold per tenant, which is why isolating delivery queues, retry state, and credentials per customer matters as much as the retry algorithm itself; we cover that boundary in our guide to multi-tenant integration architecture. And webhook delivery almost always sits downstream of an authenticated connection in the first place — see how OAuth integrations are typically built and how managed OAuth works for B2B SaaS for the piece that happens before any event ever reaches your webhook handler.

If you've already read our overview of why webhooks still matter in 2026 or the basics in what webhooks are, this is the next layer down: not whether to use webhooks, but how to make the ones you already use survive a bad network day.

This is also exactly the kind of plumbing that an integration layer is meant to absorb. Building retry schedules, dead-letter queues, per-tenant dedup stores, and reconciliation jobs for every provider you integrate with is weeks of work per connector, repeated every time a new integration gets added. viaSocket's integration layer handles delivery retries, backoff, and per-tenant isolation as part of the underlying infrastructure, so a product team wiring up a new integration inherits reliable delivery by default instead of rebuilding this pattern from scratch for every API they connect to. If you're adding integrations to a SaaS product and want a sense of how that build-vs-buy decision usually plays out, our guide on adding integrations to a SaaS product walks through the tradeoffs in more depth.

• Webhook delivery is at-least-once, not exactly-once — treat duplicates as guaranteed, not as an edge case.

• Senders retry with exponential backoff and jitter; the retry window differs wildly by provider, so check each one rather than assuming a default.

• Receivers must be idempotent: dedupe on the provider's event ID, do the check-and-record atomically, and extend the idempotency key to any downstream API calls your handler triggers.

• Accept fast and process asynchronously so the provider's retry window only has to cover acceptance, not your own processing time.

• Pair webhooks with periodic reconciliation — it's the only backstop for deliveries that never arrive at all.