Transactional outbox

Use a transactional outbox when a committed local state change must reliably cause an asynchronous publish, and the database and the broker cannot share a single commit. Write the state change and the intent to publish as one local transaction, then accept — deliberately — that the handoff to the broker is at-least-once, not exactly-once.

A database commit and a broker publish are separate failure domains. Publishing after the commit can crash before the publish happens, silently dropping the event for state that is now permanently true. Publishing before the commit can announce a change that then rolls back, telling downstream systems about something that never became real. Neither order removes the gap — only making the state change and the outbox insert one atomic transaction does.

• Business row — the state change the transaction is really about; the outbox exists to serve it, not the other way around.
• Outbox row — an immutable record of intent to publish: a stable event ID, an aggregate key, a sequence, and a payload, committed in the same transaction as the business row.
• Relay — a process that claims pending outbox rows, publishes them to the broker, and records the outcome.
• Lease — a time-bounded claim a relay worker holds on a row while publishing it, so a crashed worker's claim can eventually be taken over by another.

The transaction boundary is the entire point. A business row and its outbox row commit together or neither commits — that single fact eliminates "state committed, event never recorded" and "event intent recorded for state that rolled back." Everything past that boundary — the relay, the broker, the consumer — operates asynchronously and can fail independently, which is why the outbox does not eliminate duplicates; it only guarantees the intent to publish was never lost.

The one duplicate window the pattern cannot remove: the relay-to-broker handoff cannot be atomically committed with both systems in scope. If a relay crashes after the broker acknowledges but before the outbox row is marked delivered, the next claim republishes the same event. Marking a row delivered *before* durable broker acknowledgement is the mirror-image bug — it can lose the event outright, because nothing will ever retry a row the system already believes is done.

Ordering is per aggregate, not global. A partition key such as an aggregate ID keeps related events together where the broker's semantics support it, but it says nothing about order across unrelated aggregates or event types. A sequence number lets a consumer detect a gap or a stale update for one aggregate; it is not evidence of system-wide order.

Reach for a transactional outbox when several of the following hold:

• A committed local state change must reliably cause a downstream event — invoices, fulfilment, search projections, notifications, integration sync, or audit trails.
• The local database is authoritative for the state in question, and the service owns a transactional datastore shared with the outbox table.
• Eventual propagation to downstream systems is acceptable; the write does not need to wait for a broker round-trip to succeed.
• An unavailable broker should never force a user-facing write to fail — the business change should still be able to commit.

The outbox is not free reliability infrastructure — it trades one set of problems for another:

• Extra writes, indexes, and storage for the outbox table itself, and lifecycle management to clean it up.
• Relay capacity becomes a dependency. Someone has to run, scale, and monitor the process that turns committed intent into an actual publish.
• Propagation is delayed, bounded by poll interval, batch size, and backlog — not instant.
• Duplicate handling becomes mandatory, not optional, for every downstream consumer of these events.
• Schema and contract ownership for the event payload now sits alongside the business schema, and both evolve together.
• Dashboards, alerts, and an operational repair path are needed for pending age, retry counts, and quarantined events — without them, a stuck relay is invisible until someone downstream notices missing data.

Do not reach for a transactional outbox when:

• There is no durable downstream consequence — the event is genuinely best-effort, such as a non-authoritative usage metric.
• The operation must synchronously confirm with a downstream system before the local write can be considered complete — an outbox cannot make that wait go away.
• A direct transactional integration is available and appropriate, or a managed change-data-capture platform already gives the atomic source capture and replay controls the outbox would otherwise provide.
• Broker publication is merely best-effort telemetry, where losing an occasional message has no real consequence.
• The service does not own a transactional datastore shared with its business state — the entire premise of the pattern is absent without one.

Publishing directly, without an outbox, does not remove the dual-write problem — it just accepts the risk instead of solving it.

A merchant onboarding service stores a newly approved merchant in PostgreSQL. A merchant.approved event must trigger a separate provisioning service to set up a payment configuration, and a search-index service to make the merchant discoverable internally. The approval endpoint has a 300ms service budget; the broker can be unavailable for minutes without that being acceptable reason to fail an approval. The provisioning API charges nothing but a duplicate request can create a duplicate configuration unless it is called with the merchant ID as an idempotency key.

Flow:

1. A compliance reviewer approves the merchant. The handler begins one local transaction.
2. It updates merchants.status to approved, and inserts an outbox_events row — event UUID, aggregate key merchant:{id}, sequence, payload — in the same transaction.
3. The transaction commits. The endpoint returns success without waiting for the broker at all.
4. A relay claims a bounded batch of pending rows using row locking (FOR UPDATE SKIP LOCKED), setting a lease so other relay workers skip rows already claimed.

1. On a publish call, the relay applies a short per-attempt timeout — a few seconds, well under how long a row is allowed to stay unresolved — separate from the retention deadline for the row as a whole. A single slow attempt should fail fast and free the lease, not stall the batch. The relay classifies the outcome: a retryable failure (timeout, 5xx, broker unavailable) releases the lease and schedules another attempt; a permanent failure (malformed payload, unknown schema version) skips retries entirely and quarantines the row with an alert; an outcome-unknown failure — the call timed out but the broker may have already received it — is treated as retryable, because the outbox's own idempotent event ID makes a possible duplicate publish safe, which is exactly the case a bare direct-publish approach cannot recover from.
2. Each retry adds backoff — a capped exponential delay between attempts — with jitter layered on top so that many rows failing at once do not all retry in lockstep and overwhelm the broker on recovery. Backoff controls how long the relay waits; jitter controls how that wait is randomized so retries spread out instead of syncing.

Configuration choices: batch size starts at 50–200 rows, bounded by broker request limits and how long a lease should reasonably be held. Lease duration is set with headroom above the per-attempt timeout — 60 seconds against a typical publish well under a second. The relay tracks two separate numbers per row: retry count, how many times this particular attempt has been retried since it last failed, and total attempts, the lifetime count across every lease this row has ever been claimed under — a row can be retried a bounded number of times per lease and still be well within its lifetime attempt budget after a worker crash resets the lease. All of this — per-attempt timeout, backoff, and the number of attempts — has to fit inside the row's own remaining age budget: an outbox row that has been pending for close to its maximum acceptable age should stop taking on new backoff delays and instead move to quarantine, the same way a request retry has to fit inside its remaining end-to-end deadline rather than being retried indefinitely.

Before shipping an outbox-backed publish path, confirm:

• The business write and the outbox insert commit in the same local transaction — never as two separate writes.
• Every outbox row carries a stable event ID used for downstream deduplication, not a fresh ID per publish attempt.
• The relay marks a row delivered only after durable broker acknowledgement, never before.
• Concurrent relay workers use a lease or row-locking claim protocol, with an expiry that allows recovery after a crash.
• Retryable, permanent, and outcome-unknown failures are classified explicitly, with quarantine and alerting for permanent failures.
• Per-attempt timeout, backoff, and jitter are configured for relay retries, and both retry count and lifetime total attempts fit inside the row's maximum acceptable pending age.
• Downstream consumers have an explicit idempotency boundary — an event ID, a natural key, or both.
• A partition key and, if needed, a per-aggregate sequence are chosen deliberately, with no assumption of order beyond that key.
• Pending count and age of the oldest pending row are both monitored, with alerting tied to the age, not just the count.
• Retention and cleanup for delivered rows are defined, balancing audit and replay needs against table growth.