How to audit whether a webhook was ever received

Someone from finance asks whether a particular Stripe event arrived on the 14th. A customer insists an order never came through. An auditor wants six months of payment events.

These questions all arrive weeks after the fact, and they all have the same answer if you never stored anything: probably, but we can't prove it.

Receiving a webhook takes an hour or two. Being able to answer questions about it three weeks later is a different job, and it is the one that tends to get skipped.

Why the gateway's log isn't enough

If you use a gateway or the provider's own dashboard, you do get a delivery log. It is genuinely useful while you are building, and it is the first place to look when something breaks today.

It has two limits. Retention windows are short, days or weeks rather than the years an audit question tends to reach back over, so the log has often expired by the time finance asks. And it records delivery rather than outcome: it can tell you a request returned 200, but not whether your code then failed to create the order.

"Was it delivered?" and "what happened to it?" are different questions. The second one needs your own record.

What should you store?

That is more than the payload alone, and a good deal less than everything the request contained.

• The provider's event ID. This is what makes an audit possible. It is the thing finance or the provider's support team will quote at you.
• The event type, so you can query by category later.
• A received timestamp of your own, separate from any timestamp inside the payload.
• The full raw payload. Storage is cheap and you will not anticipate which field matters in six months.
• Processing status, and the error if it failed. This is the field that answers "what happened to it", which is the whole point.

Store the event before you process it. If you store it afterwards and your processing crashes, you have lost the record of the thing you most need to investigate.

Storing the event

Two details in this handler matter more than they look.

Inbound routes need an explicit auth bypass, because routes require a key by default and the provider does not have one. And signature verification has to run against req.rawBody, the unparsed body, because re-serialised JSON will not produce a matching signature.

Verification itself is worth not writing by hand. Every provider signs differently (Stripe includes a timestamp, Shopify base64-encodes, Discord uses Ed25519), and getting one of those details wrong gives you a check that passes when it should not. We maintain webhook-verify for this: one verify() call covering 21 providers, MIT licensed with no dependencies.

import { app, Datastore } from 'codehooks-js';
import { verify } from 'webhook-verify';

// Providers cannot send an API key, so let this route through
app.auth('/webhook/*', (req, res, next) => next());

app.post('/webhook/stripe', async (req, res) => {
  // Always the raw body, never the parsed one
  if (!verify('stripe', req.rawBody, req.headers, process.env.STRIPE_WEBHOOK_SECRET)) {
    return res.status(401).json({ error: 'Invalid signature' });
  }

  const db = await Datastore.open();
  const event = req.body;

  // Already seen it? Acknowledge and stop. findOneOrNull rather than
  // findOne, which throws when nothing matches.
  const seen = await db.findOneOrNull('events', { eventId: event.id });
  if (seen) {
    return res.status(200).json({ received: true, duplicate: true });
  }

  // Store first, process second
  await db.insertOne('events', {
    eventId: event.id,
    type: event.type,
    receivedAt: new Date().toISOString(),
    status: 'received',
    payload: event
  });

  await db.enqueue('process-event', { eventId: event.id });

  // Acknowledge quickly so the provider stops retrying
  res.status(200).json({ received: true });
});

app.worker('process-event', async (req, res) => {
  const db = await Datastore.open();
  const { eventId } = req.body.payload;

  try {
    await handleEvent(eventId);
    await db.updateOne('events', { eventId }, { status: 'processed' });
  } catch (err) {
    // Record the failure rather than losing it. Nothing retries a
    // throwing worker for you, so there is no reason to re-throw:
    // the record is the retry hook.
    await db.updateOne('events', { eventId }, {
      status: 'failed',
      error: String(err)
    });
  }
  res.end();
});

export default app.init();

The handler stores and acknowledges, then a worker does the actual work. That split matters for auditing: the provider gets its 200 quickly and stops retrying, and if your processing fails, the event is already recorded along with the reason.

What about the same event arriving twice?

It will happen. Providers retry when they don't get a timely acknowledgement, including when your handler succeeded but the response was slow. So the same event arrives again, and if you process it twice you create two orders.

The check above handles this by looking for the event ID before doing anything. That is enough for most cases, and it works because providers send a stable ID per event rather than per delivery attempt.

Be aware of the race: two copies arriving simultaneously can both pass the check before either inserts. If double-processing would be expensive, make the downstream operation idempotent as well, keyed on the event ID, so a repeat is harmless rather than merely unlikely.

Answering the question

This is the part that makes the storage worth having. Once events are in a queryable database, an audit is a query.

// Everything received in a date range, newest first
app.get('/events', async (req, res) => {
  const db = await Datastore.open();
  const { from, to, type } = req.query;

  const query = { receivedAt: { $gte: from, $lte: to } };
  if (type) query.type = type;

  db.getMany('events', query, { sort: { receivedAt: -1 } }).json(res);
});

// Everything that failed, so it can be replayed
app.get('/events/failed', async (req, res) => {
  const db = await Datastore.open();
  db.getMany('events', { status: 'failed' }).json(res);
});

Someone can now answer "did we receive that event on the 14th" with a single request, instead of going digging. Replaying a failed event is a matter of re-queueing it, using the payload you already stored.

How long should you keep them?

Longer than you think, because the storage is cheaper than the alternative.

Do the arithmetic for your own volume. At 2,000 events a day you accumulate around 730,000 records a year. That is a lot of rows and not much data, and it is considerably less work than reconstructing a payment history from two systems that no longer agree.

A reasonable default is to keep the full payload for as long as your audit or dispute window requires, then trim the payload while keeping the ID, type, timestamp and status. That keeps "did we receive it" answerable indefinitely at a fraction of the size.

Summary and conclusion

The gateway log tells you a request was delivered. Your own records tell you what happened to it afterwards, and that is what the questions weeks later are actually about.

Store the provider's event ID, the type, your own timestamp, the raw payload and a processing status. Store it before you process it, check the event ID so a retry is harmless, and put the processing in a worker so a failure is recorded rather than lost.

If you want the whole picture, with the database, queues and cron sitting behind every endpoint, that is what Codehooks is built for.