The Transactional Outbox — a Sticky Note the Crash Can't Kill

The dual-write post ended on a sticky note: if you must save a file and send a message, don’t trust your post-crash self to send it — in the same motion as the save, stick a note on the desk, and let your roommate deliver it. The outbox pattern is that sticky note, built out of the one tool that already knows how to survive crashes: your own database.

The entire trick: instead of publishing to the broker, write the event as a row into an outbox table in the same transaction as the data — both rows or neither — and let a separate relay read the table and publish, retrying until every row is delivered.

The sticky-note table#

One new table. It’s the desk the sticky notes go on:

CREATE TABLE outbox (
    id           UUID PRIMARY KEY,
    event_type   TEXT        NOT NULL,   -- 'OrderPaid'
    payload      JSONB       NOT NULL,   -- {"orderId": "4211", ...}
    created_at   TIMESTAMPTZ NOT NULL DEFAULT now(),
    published_at TIMESTAMPTZ             -- null = not delivered yet
);

The business code stops talking to the broker entirely. Both writes are now rows in the same database, so one ordinary transaction genuinely covers them:

@Transactional
public void markPaid(Order order) {
    order.pay();
    orders.save(order);                                      // the fact
    outbox.save(new OutboxEvent("OrderPaid", order.id()));   // the sticky note
}   // both rows committed together, or neither — and the request is done

And the roommate — a relay that runs forever in the background, on its own clock, in its own process or thread:

@Scheduled(fixedDelay = 500)
public void drainOutbox() {
    List<OutboxRow> batch = outbox.lockNextBatch(100);
    for (OutboxRow row : batch) {
        broker.publish(row.eventType(), row.payload());
        outbox.markPublished(row.id());
    }
}

Plain-words walkthrough:

• markPaid never meets the broker. Its whole job is two INSERTs into one database. If the process dies one nanosecond after commit, nothing is lost — the note is a row now, and rows don’t die with processes.
• The relay’s whole life is a loop: find rows where published_at is null, publish them, stamp them. If the broker is down, the publish throws, the row stays unstamped, and the next tick tries again. The retry plan finally lives outside the thing that crashes.
• lockNextBatch hides one production-grade line of SQL — worth seeing once:

SELECT * FROM outbox
WHERE published_at IS NULL
ORDER BY created_at
LIMIT 100
FOR UPDATE SKIP LOCKED;

FOR UPDATE locks the rows this relay instance is working on; SKIP LOCKED makes a second relay instance skip those rows instead of waiting. That one clause is what lets you run three relays for redundancy without them fighting over — or double-publishing — the same rows at the same time.

Here’s the happy path, end to end — notice it’s two independent stories:

sequenceDiagram
  participant S as markPaid()
  participant DB as database
  participant R as relay
  participant B as broker
  S->>DB: one transaction: orders row + outbox row
  DB-->>S: committed
  Note over S,DB: the request's story ends here
  R->>DB: any unpublished rows?
  DB-->>R: row 17: OrderPaid #4211
  R->>B: publish(OrderPaid)
  B-->>R: ack
  R->>DB: stamp row 17 published

The fine print: at-least-once, and where duplicates are born#

The relay has its own tiny gap, and it’s worth watching closely, because this is the exact birthplace of the duplicates that the whole idempotency discipline exists for:

That’s the contract this pattern signs, and every honest description says it out loud: at-least-once delivery. Never zero — the row can’t be lost — but occasionally two, because “publish” and “stamp the row” are themselves two steps. (Yes: the pattern that fixes a dual write contains a miniature dual write. The difference is the failure direction — this one duplicates instead of losing, and duplicates can be handled: every consumer checks have I seen this event id before? — the elevator-button move from the problem post, which is its own pattern with its own notes.)

Two more honest costs. The event arrives a little later than the commit — half a second of relay tick, usually invisible, occasionally the thing you have to explain to a product manager. And the outbox table grows forever unless you clean it: stamped rows need a periodic delete (keep a few days for debugging), and a monitor on the count of unstamped old rows — that number climbing is your “the relay is stuck” alarm.

In the wild#

• microservices.io lists it as the canonical fix for exactly this problem — Transactional Outbox is the pattern’s official name in the catalog.
• Debezium ships a built-in “outbox event router” because so many teams do exactly this — that’s the tailing variant of the relay, covered in the change data capture notes.
• Spring Modulith’s externalized events persist application events in a table and republish after crashes — an outbox wearing framework clothes.
• Every team that ever wrote a pending_notifications table with a cron job that sends and marks rows — that was an outbox, discovered independently, probably without the name.

When the outbox is overkill#

• The event doesn’t matter enough. Metrics, page-view pings, cache warm-ups — if losing one is fine, publish directly and skip the machinery. The outbox tax is only worth paying for events someone downstream must receive.
• You’re already running CDC infrastructure. If Debezium is tailing your database anyway, use it as the relay instead of writing a polling loop — same guarantee, less code to own.
• There’s no second system. If the “event” is consumed inside the same process and database, an ordinary table and a transaction already give you everything — no broker, no problem, no pattern.