Outbox Worker

Requirement A durable-delivery specification MUST document delivery guarantee, downstream boundary, duplicate behavior, retry horizon, recovery owner, record owner, and status.

Rationale The specification defines the expected delivery behavior before a message record exists. A draft names its decider and decide-by date.

Requirement A project MUST NOT activate outbox for a reaction whose delivery guarantee is best-effort and which operators can repeat manually.

Rationale Outbox storage and dispatcher operation are unnecessary when loss is an accepted recoverable outcome. The delivery guarantee comes from the reaction's own declaration. The module specification records it with the vocabulary its event table already uses.

Requirement A reaction the outbox excludes MUST publish after its commit succeeds without staging an outbox record.

Rationale Best-effort delivery has no durable record. The commit success is its only guarantee.

Requirement A command post-handler MUST stage pending durable messages in the same scoped business session as its aggregate changes.

Rationale One session makes the business commit and outgoing message durable together.

Requirement An outbox writer MUST serialize each pending durable message into an outbox record.

Rationale A durable record holds the dispatch input after the command scope completes.

Requirement An outbox record MUST contain message identity, kind, stable type, schema version, time, payload, tenant, state, attempt, and next-attempt data.

Rationale Dispatch and recovery need one complete durable description of the message.

Requirement An outbox record MUST retain its last error summary and completion time.

Rationale Operators need failure and completion evidence for backlog review and replay.

Requirement An outbox record MUST name its message kind as integration event, journey-step command, or provider side effect, with an integration event citing its Event record.

Rationale The kind decides dispatch ownership and transport. The Event record carries the raises and reacts-to links.

Requirement An outbox writer MUST NOT use another connection or independent commit for required durable delivery.

Rationale A shared PostgreSQL database does not make separate transactions atomic together.

Requirement A Worker MUST claim pending outbox records, dispatch a bounded batch, and record success or retry information.

Rationale Worker ownership separates durable dispatch from the request transaction.

Requirement WebApi MUST NOT dispatch a durable message inside the request transaction.

Rationale The request commit only makes the message eligible for later Worker dispatch.

Requirement An outbox claim MUST use a short transaction with the lease owner, lease expiry, and fencing value that litespec/ext/jobs#store-execution-leases defines.

Rationale Claim fields establish which Worker owns one dispatch attempt.

Requirement A Worker MUST NOT hold an outbox claim transaction open during a network call.

Rationale Network delay does not retain database locks or transaction resources.

Requirement A Worker MUST update an outbox record only while it owns the matching lease and fencing value.

Rationale A stale Worker does not overwrite a newer owner's dispatch result.

Requirement An outbox store MUST return expired claims to eligible work.

Rationale A crashed Worker cannot permanently strand a pending message.

Requirement An outbox consumer MUST handle the same message more than once.

Rationale A crash after external acceptance can cause one message to be delivered again.

Requirement An outbox consumer MUST use message ID or a business idempotency key at its Command or side-effect boundary.

Rationale One stable identity lets the target suppress a repeated delivery.

Requirement A Worker MUST NOT mark an outbox record complete before the target boundary accepts it.

Rationale Completed state claims delivery that has not yet reached the required target.

Requirement An outbox dispatcher MUST use bounded exponential backoff with jitter.

Rationale Bounded jitter reduces repeated load during a transient target failure.

Requirement An outbox dispatcher MUST classify permanent failures, stop automatic retry at its configured limit, and retain safe investigation and replay data.

Rationale A dead-letter outcome preserves evidence after automatic recovery is no longer safe.

Requirement A dispatcher MUST distinguish an unavailable outbox store from a failed claimed message.

Rationale Store availability and a claimed record's target result require different recovery paths.

Each has its own signal. The store's availability is a gauge the readiness endpoint reads. A claimed message's outcome is a counter carrying the result, following the OpenTelemetry metrics specification. A single error counter cannot tell an operator whether to restore a database or investigate one destination.

Requirement A Worker MUST apply bounded backoff and rate-limited logging when its outbox store is unavailable.

Rationale Cold start before schema application and store outage do not justify one exception per loop.

Requirement A Worker MUST apply message retry and poison handling only to claimed outbox records.

Rationale An unavailable store has no claimed message eligible for delivery retry.

Requirement A Worker host exposing readiness MUST gate dispatch and stop polling stores that cannot serve requests.

Rationale Readiness prevents avoidable work against unavailable storage dependencies.

Requirement An outbox message MUST use a stable type name and explicit schema version.

Rationale The pair defines a durable contract for Worker deserialization and dispatch.

Requirement A deployed dispatcher MUST process records from every application version that can coexist during rollout or rollback.

Rationale Active deployment versions can place different valid message shapes in one store.

A message record therefore carries its own shape version. The dispatcher selects a reader from that value. Adding an optional member with a defined absent-value behavior needs no new version. Removing a member, renaming one, narrowing a type, or changing a discriminator needs a new version. The older reader stays until no record carries the older version.

Requirement An outbox deployment MUST publish pending count, failed count, oldest pending age, attempts, dispatch duration, and success rate.

Rationale These indicators make backlog growth and dispatch quality observable.

Requirement An outbox deployment MUST set alert thresholds as Limit records from its use-case delivery target.

Rationale Alert urgency follows the business consequence of delayed delivery.

Requirement An outbox record MUST model its lifecycle as an abstract state base with one sealed record for each of pending, processing, dispatched, and dead-letter.

Rationale pending becomes processing. processing becomes dispatched, pending, or dead-letter. Audited replay moves dead-letter to pending. Three of the four carry data the others do not. processing holds a lease expiry. dispatched holds a dispatch time. dead-letter holds a failure reason and an attempt count. A four-value label forces those onto nullable columns that are meaningful in one state each, which is the shape litespec/backend/domain#model-every-closed-set-of-domain-values-as-a-closed-type-hierarchy removes everywhere else.

Example OutboxState.Processing carries one unexpired lease before a dispatcher records OutboxState.Dispatched or a retry state.

Requirement An outbox dispatcher MUST preserve message ID across retry and replay.

Rationale Stable identity lets target idempotency suppress delivery after a crash before dispatch completion.

Requirement An outbox replay MUST record actor, reason, time, and failure without editing the payload: an audit record when audit is selected, a standalone record otherwise.

Rationale Replay history preserves message evidence and explains a renewed dispatch attempt.

Requirement New WebApi code MUST emit a message only after every active Worker can read its type and schema version.

Rationale A writer does not create a message shape that a running dispatcher cannot understand.

Requirement A release MUST deploy compatible readers before writers for additive event versions.

Rationale Reader-first deployment preserves dispatch during a mixed-version rollout.

Requirement A rollback Runbook MUST include records created by the new writer but not yet dispatched.

Rationale These records can outlive the deployment that originally wrote them.