Webhook Testing and Debugging Guide: Verify Signatures, Retries, and Duplicate Events
WebhooksAPIsDebuggingBackend DevelopmentIntegrationsDeveloper Tools

Webhook Testing and Debugging Guide: Verify Signatures, Retries, and Duplicate Events

SScrapes.us Editorial Team
2026-11-21
8 min read

A practical webhook testing guide covering local capture, signatures, retries, duplicate events, replay fixtures, and recurring health checks.

Webhook testing is easier when you separate delivery, authentication, payload handling, retries, and business effects. This guide provides a repeatable workflow for testing webhooks locally, diagnosing rejected or duplicate events, verifying signatures safely, and monitoring the integration after deployment.

Overview

A webhook is an HTTP request sent by one system when an event occurs in another system. The receiving application must accept the request, determine whether it is authentic, parse the payload, process the event, and return an appropriate response. A failure at any step can look similar from the outside: the provider may retry, the application may create duplicate records, or an event may appear to disappear.

Effective webhook testing starts by making each part observable. Record the request method, URL, status code, headers, raw body, response time, provider event ID, and processing result. Do not rely only on the parsed JSON body. Signature verification often depends on the exact bytes received, including whitespace and encoding.

A useful local-testing workflow is:

  1. Expose a local endpoint through a secure development tunnel or use a request-inspection endpoint.
  2. Send a harmless test event from the provider dashboard or a command-line tool.
  3. Capture the complete request, including headers and the raw body.
  4. Compare the request with the provider’s documented format and your expected schema.
  5. Test valid, invalid, delayed, repeated, and out-of-order events.
  6. Replay a captured request against a local or staging handler.
  7. Confirm that logs and metrics identify the event without exposing secrets or sensitive data.

Choosing a tool for the job

Tool typeBest useLimitations to check
Secure tunnelDelivering provider requests to a local applicationURL changes, access controls, and whether the tunnel preserves headers and methods
Request-inspection serviceViewing headers, body, and delivery attempts before writing codeDo not send production secrets or personal data; it may not reproduce your application logic
Mock serverReturning controlled status codes and testing retriesIt may not model provider-specific signing or event ordering
Provider dashboardSending official test events and reviewing delivery historyAvailable event types, replay controls, and displayed details vary by provider

Use the smallest tool combination that answers the current question. A request-inspection endpoint is useful for confirming what arrives. A tunnel is better when you need to exercise your own parser, signature verifier, database, and queue.

What to track

1. Delivery and HTTP behavior

For every delivery attempt, track the timestamp, provider event ID, event type, endpoint, HTTP method, response status, response body category, and processing duration. A successful HTTP response only proves that the endpoint responded as expected; it does not necessarily prove that the business operation completed. If your application acknowledges before asynchronous processing, record the queue or job ID as a separate field.

Verify basic HTTP assumptions during local testing:

  • The endpoint accepts the method used by the sender, normally POST.
  • The content type is handled explicitly, rather than assumed.
  • The application reads the request body once and passes the raw bytes to signature verification.
  • Authentication and authorization failures are distinguishable from application errors in logs.
  • Response codes match the integration’s documented retry expectations.

2. Signature verification

Most signed webhooks use a shared secret and a message authentication code, often calculated over the raw body and sometimes a timestamp. The exact header names, separators, algorithms, and comparison rules are provider-specific, so treat the provider’s documentation and SDK as the authority.

A generic HMAC verification model looks like this:

signedMessage = timestamp + "." + rawBody
expected = HMAC_SHA256(webhookSecret, signedMessage)
valid = constantTimeCompare(expected, signatureFromHeader)

This is a pattern, not a drop-in implementation. Some systems sign only the body, use a different digest, include a version prefix, or send multiple signatures during secret rotation. Keep the secret outside source control, use constant-time comparison where supported, and reject malformed or missing signatures before performing business actions.

When debugging a rejected signature, log safe diagnostic facts such as the event ID, algorithm label, timestamp presence, and a hash or length of the body. Avoid logging the secret, full authorization headers, or sensitive payload fields. Check whether a JSON parser, middleware, proxy, or character conversion changed the body before verification.

3. Timestamps and replay protection

If a signature includes a timestamp, compare it with the server’s clock using a documented tolerance appropriate for the integration. A large clock difference can make valid requests fail. A timestamp check should complement, not replace, signature verification. Store a short-lived record of accepted event IDs or signed request identifiers when replay protection is required.

4. Payload and schema behavior

Record which fields are required, optional, nullable, or version-dependent. Test unknown fields and missing optional fields so a harmless provider addition does not break parsing. Validate the payload before writing records or triggering irreversible actions. A JSON Schema workflow can help formalize these checks; see the JSON Schema Validation Guide for a related validation process.

Keep a small fixture set under version control, with secrets and personal data removed:

fixtures/
  payment-created.json
  payment-created.headers.json
  payment-created.invalid-signature.json
  payment-created.duplicate.json
  payment-created.out-of-order.json

5. Retries and duplicate events

Assume that delivery can be repeated. A timeout, connection failure, non-success response, or provider-side interruption may cause another attempt even when your application began processing the first request. The receiver should be idempotent: processing the same event more than once should not create duplicate side effects.

A common approach is to store the provider’s event ID in a durable table with a uniqueness constraint. At the start of processing, attempt to claim the event. If it already exists, return a safe acknowledgment or follow the integration’s documented duplicate behavior. Be careful with a simple “seen” flag: if a worker crashes after claiming an event but before completing the action, you need a recovery state, lease, or retryable job design.

receive event
  verify signature and timestamp
  validate event shape
  claim event ID atomically
  if already completed: acknowledge duplicate
  if claimed: enqueue or process safely
  record completed or retryable failure

For integrations that can deliver related events out of order, use the resource version, sequence value, or event timestamp when the provider supplies one. Do not assume that arrival order equals occurrence order.

Cadence and checkpoints

Webhook reliability is not a one-time deployment task. Track a small set of checkpoints monthly or quarterly, depending on how often the integration changes:

  • Delivery health: successful responses, rejected requests, timeouts, and processing latency.
  • Retry volume: repeated event IDs, attempts per event, and events that require manual recovery.
  • Signature health: invalid signatures, missing headers, timestamp failures, and secret-rotation test results.
  • Schema drift: new event types, changed fields, unknown properties, and parser warnings.
  • Business idempotency: duplicate records, repeated emails, duplicate payments, or other repeated side effects.
  • Operational readiness: alert routing, dashboard usefulness, replay procedures, and access to sanitized fixtures.

At each checkpoint, replay at least one known-good fixture and one failure fixture in a non-production environment. Confirm that the handler produces the expected status, log fields, queue behavior, and database result. If your application uses scheduled recovery jobs, document their timing and test them alongside the webhook. The Cron Expression Guide can help when reviewing scheduled retry or reconciliation tasks.

Re-run the full checklist after changing the endpoint URL, framework middleware, request-body parser, queue, database constraint, webhook secret, or event schema. These changes can affect verification and idempotency even when the handler’s business code appears unchanged.

How to interpret changes

A sudden increase in duplicate events does not automatically mean the provider is malfunctioning. First compare duplicate volume with response latency, timeout logs, deployment times, and database contention. If duplicates appear after slow processing, acknowledge-and-queue or improve the idempotency boundary rather than merely suppressing logs.

An increase in invalid signatures points to a different class of problem. Check for a rotated secret, a changed signing scheme, a body parser that runs too early, proxy modifications, clock drift, or a test request that was manually reconstructed instead of replayed byte-for-byte. Compare a captured request with a verified fixture before changing cryptographic code.

A rise in unknown event types may be normal provider expansion, but it should not silently trigger unintended actions. Route unsupported events to a visible, safely acknowledged path while you decide whether to add support. A rise in schema validation failures often indicates a contract change, malformed test data, or an assumption that an optional field is always present. Contract testing and API linting provide useful complementary practices; see the OpenAPI Linting and Contract Testing Guide.

Interpret metrics in context. A high request count with stable successful processing may simply reflect increased usage. A low request count with no successful events may indicate a disabled subscription, incorrect endpoint, or authentication failure. Always compare provider delivery records with receiver logs so you can distinguish “not sent,” “not received,” “rejected,” and “accepted but not completed.”

When to revisit

Review this webhook testing checklist on a monthly or quarterly cadence, and immediately after an integration change. Revisit it when a provider adds an event version, changes its signature format, introduces a new retry behavior, or recommends a new SDK. Also review it after rotating secrets, migrating frameworks, changing reverse proxies, moving from synchronous processing to a queue, or observing unexplained duplicates.

Use this practical sequence before every significant webhook release:

  1. Create or refresh a sanitized fixture from a representative event.
  2. Replay it locally and confirm raw-body signature verification.
  3. Replay an invalid-signature request and confirm that no business action occurs.
  4. Send the same valid event twice and verify idempotent behavior.
  5. Simulate a timeout or temporary failure and confirm safe retry handling.
  6. Test a missing optional field and an unknown field.
  7. Check logs, alerts, queue state, and reconciliation records.
  8. Record the provider documentation version or review date alongside the integration notes.

Keep the checklist, fixtures, and replay instructions near the service code. When an incident occurs, you should be able to reproduce the request without searching through production logs or exposing secrets. For broader API troubleshooting, the Developer Debug Toolkit offers related techniques for inspecting responses, decoding tokens safely, and formatting structured data.

Related Topics

#Webhooks#APIs#Debugging#Backend Development#Integrations#Developer Tools
S

Scrapes.us Editorial Team

Developer Tools Editor

Senior editor and content strategist. Writing about technology, design, and the future of digital media. Follow along for deep dives into the industry's moving parts.