Scheduled Jobs

Requirement A scheduled-job record MUST name schedule, time zone, delay, overlap, misfire, idempotency, retry, owners, and status, with a draft naming decider and decide-by date.

Rationale The contract describes when work may run and how it recovers from normal scheduling faults. Owners are the operating owner and the record owner.

Requirement A scheduled job MUST run in Worker outside WebApi request handling.

Rationale Worker execution permits independent scheduling and controlled replica shutdown.

Requirement A scheduled job MUST observe cancellation, have bounded duration, and stop accepting new work during graceful shutdown.

Rationale Bounded work lets a Worker stop without abandoning uncontrolled execution.

Requirement A scheduled-job store MUST record lease owner, expiry, and fencing value in PostgreSQL.

Rationale The lease identifies the replica that can complete one planned occurrence. The outbox extension cites this definition for its own claims.

Requirement A Worker replica MUST renew only its active lease and stop when renewal fails.

Rationale Losing the lease ends authority to complete the scheduled occurrence.

Requirement A scheduled-job lease MUST prevent two replicas from completing the same occurrence.

Rationale One occurrence needs one logical completion even when replicas overlap.

Requirement A scheduled-job store MUST generate each fencing value monotonically per occurrence and check it on every completion write.

Rationale A monotonic value lets the store reject a completion from a Worker that no longer owns the lease.

Requirement A scheduled-job occurrence and its lease MUST carry tenant scope when the product isolates tenants.

Rationale Tenant scope keeps one tenant's scheduled work separate from another's.

Requirement A scheduled job MUST dispatch an Application Command or approved Application port.

Rationale Application owns the use-case boundary that the schedule triggers.

Requirement A scheduled job MUST NOT mutate a Marten session or DbContext directly, bypass Domain behavior, or call WebApi endpoints.

Rationale Direct storage and HTTP calls bypass the command boundary and its authorization, transaction, and validation behavior.

Requirement A scheduled job MUST use bounded attempts, cancellation-aware backoff, and a poison state for permanent failure.

Rationale Bounded retries prevent one failed occurrence from consuming Worker capacity indefinitely.

Requirement A scheduled job Command and side effect MUST tolerate duplicate execution after lease or process failure.

Rationale A failed process can leave uncertain completion before its lease expires.

Requirement A Worker MUST treat an unavailable scheduler store as the store outage litespec/ext/outbox#distinguish-dependency-outage defines, with backed-off, rate-limited logging.

Rationale Store availability differs from a claimed occurrence's retry or poison outcome.

Requirement A scheduled-job store MUST persist each occurrence's last scheduled, started, and completed time.

Rationale Persisted points identify missed work and completed work after restart.

Requirement A scheduled-job specification MUST define skipped, once-replayed, or interval-replayed behavior for missed occurrences.

Rationale The selected policy controls the business effect of downtime and delay.

Requirement A scheduled-job owner MUST provide Runbook records for inspection, manual replay, and disable.

Rationale Operators need controlled actions for stalled, failed, or unsafe schedules.

Each procedure names the interface that performs it. A procedure that says "disable the schedule" without naming a command, an endpoint, or a configuration value remains undiscovered until the incident it was written for. Avoid an interface that writes to the schedule store directly, because it bypasses the occurrence identity litespec/ext/jobs#derive-stable-occurrence-identity depends on.

Example A disable procedure names the configuration value or the administrative endpoint that stops the schedule. It states what happens to an occurrence already claimed.

Requirement A scheduled-job store MUST derive each occurrence identity from its schedule ID and scheduled instant, removing occurrences older than the schedule's retention Limit record.

Rationale One planned time receives one stable identity across retries and replicas.

The key space grows with the schedule's frequency and its retention. A per-second schedule kept for a year holds more than 30 million rows. Quartz states the same practice for a persistent job store.

Example A daily schedule that retains occurrences for 90 days holds 90 rows. A minute schedule with the same retention holds more than 100,000 rows. That is a different storage decision.

Requirement A scheduled-job store MUST persist scheduled, processing, completed, and dead-letter state with safe execution details.

Rationale State records include attempt, lease, start, completion, and safe error data.

Requirement A scheduled job MUST use its occurrence identity as the command idempotency scope.

Rationale Retries of one occurrence remain tied to the same scheduled business action.

Requirement A Worker MUST mark completion only while its fencing value remains current.

Rationale A stale Worker does not complete work after another replica owns the occurrence.

Requirement A manual replay MUST create a new attempt for the same occurrence identity: an audit record when audit is selected, a standalone record otherwise.

Rationale Replay preserves the original scheduled instant and its history.

Requirement A scheduled job MUST use a UTC schedule unless its specification names a business-local time requirement and its IANA time zone.

Rationale A local schedule runs twice on one date each year and skips an hour on another. A daylight-saving transition does that to a wall clock. That behavior is correct for a job whose meaning is local, such as a nightly statement at 9 AM for a reader. It is a defect for every other job. Naming the time zone in the specification makes the transition behavior reviewable.

Requirement A business-local schedule MUST record an IANA time-zone identifier and define skipped and repeated local-time behavior.

Rationale Clock transitions can omit or repeat a local wall-clock time.

Requirement A scheduled-job test MUST use an injected clock and representative time-zone transitions.

Rationale A controllable clock exercises delayed and daylight-saving behavior deterministically.

Requirement Scheduled-job production code MUST NOT depend on machine-local time.

Rationale Host-local configuration makes schedule behavior differ between environments.