Idempotency, locks and retry.

Operational safeguards reduce duplicate processing and accidental concurrent writes. They complement the orchestrator; they do not replace correct job design.

Idempotency policies

Policy	Behavior
always_run	Execute every time. This is the default.
skip_if_success	Skip when the same key already succeeded.
fail_if_success	Fail when the same key already succeeded.
rerun_if_failed	Run only when the previous execution for the key failed.

Example

YAML

idempotency_key: "orders:${run_date}"
idempotency_policy: skip_if_success
lock_enabled: true
retry_attempts: 5
retry_backoff_seconds: 10

Python

result = ingest(
    ...,
    idempotency_key="orders:${run_date}",
    idempotency_policy="skip_if_success",
    lock_enabled=True,
    retry_attempts=5,
    retry_backoff_seconds=10,
)

Locks

Locks are cooperative and best-effort. They protect ContractForge writers from stepping on each other, but cannot stop an unrelated Spark job from writing to the same target.

• Use locks for scheduled jobs writing the same target.
• Keep lock messages actionable for operators.
• Release happens in failure paths through the orchestration finally block.

Retry

Retry is useful for transient Delta conflicts and infrastructure noise. It should not hide deterministic contract failures such as invalid schema, bad credentials or broken quality gates.