Contracts are the source of truth.

ContractForge separates data movement, catalog metadata, operational ownership, security rules and platform environment choices into dedicated contracts. Use a single ingestion file for small workloads, or a bundle when different teams need different review and application workflows.

Contract files

ContractForge contracts are intentionally split by responsibility. The split is not mandatory for every small workload, but it becomes important when data engineering, governance, operations and security have different owners, review cycles or runtime privileges.

*.ingestion.yaml

Ingestion

Executable source, target, write mode, transforms, quality rules, schema policy, execution windows and runtime behavior.

*.annotations.yaml

Annotations

Catalog-facing descriptions, aliases, tags, PII classification and lifecycle metadata for tables and columns.

*.operations.yaml

Operations

Ownership, support groups, criticality, expected frequency, freshness SLA, runbook and dashboard tags.

*.access.yaml

Access

Grants, row filters, column masks and access drift policy, applied by a workflow with the right catalog privileges.

*.environment.yaml

Environment

Adapter selection, evidence location, runtime hints and adapter-owned deployment settings.

project.yaml

Project YAML

Repository-level environments, reusable connections, schedules, execution order and adapter deployment blocks.

The ingestion contract is the only file required to move data. The other files make catalog metadata, operational ownership, security policy and runtime environment choices explicit without forcing a table rewrite.

Reusable connection YAMLs and logical table refs reduce platform drift between adapters. A bronze JDBC dataset can inherit endpoint/auth/driver settings from source.type: connection, while downstream silver and gold contracts can read earlier ContractForge tables through source.ref: bronze.table_name or SQL {{ table_ref:layer.table }} placeholders.

The ingestion file is the bundle root and owns the physical table identity. Companion files normally omit target; ContractForge applies annotations, operations and access to the matching ingestion target. If a companion file declares target, it is treated as an explicit compatibility assertion and must match the ingestion target exactly.

Bundle layout

contracts/
  silver/
    s_customers.ingestion.yaml
    s_customers.annotations.yaml
    s_customers.operations.yaml
    s_customers.access.yaml
    s_customers.environment.yaml

A bundle is loaded by path prefix. For example, contracts/silver/s_customers can resolve the ingestion, annotations, operations and access files shown above. This keeps related decisions close in source control while allowing each file to be reviewed and applied independently.

When to split contracts

Change type	Contract	Reason
Source, target, mode, keys, quality, transform or execution behavior	ingestion	It changes how data is read, shaped, validated or written.
Description, aliases, tags, PII or deprecation metadata	annotations	It describes the table or fields without changing the data pipeline.
Owner, support group, criticality, SLA, runbook or alert intent	operations	It changes how the table is operated and monitored.
Privileges, row filters or masks	access	It requires security-oriented review and often elevated catalog privileges.
Adapter, runtime, evidence location or deployment hint	environment	It selects where and how the contract is planned or executed without changing source/target semantics.

Application model

Required for execution

Ingestion

Validated before Spark work starts. It defines what is read, how rows are prepared, which quality gates apply and how the target is written.

Applied after table exists

Annotations

Applied after the write path has created or updated the target schema. Column comments, tags and PII metadata require the physical columns to exist.

Recorded for operations

Operations

Recorded with run evidence so dashboards can show owner, criticality, support group, SLA and runbook context without parsing YAML manually.

Often privileged

Access

Designed for a dedicated workflow because grants, row filters and masks commonly require stronger catalog privileges than normal ingestion jobs.

Adapter-owned

Environment

Keeps platform selection, evidence storage and runtime hints separate from the ingestion, annotations, operations and access semantics.

Choosing single-file or bundle mode

Use a single ingestion contract when one team owns the source, target, quality rules and operational response. Use a bundle when contract changes need different approvals or when metadata/security should be updated without re-running a heavy ingestion.

Define execution behaviorDescribe tables and fieldsRecord ownership and SLAsManage access policySelect adapter environment

Shared metadata

Every contract file may include _metadata. ContractForge strips it before execution and keeps it for audit and bundle checks.

_metadata:
  contract_version: 1.0.0
  last_updated_by: data-platform
  last_updated_at: "2026-05-17T12:00:00Z"

Validation workflow

contractforge validate contracts/silver/s_customers.ingestion.yaml
contractforge validate-bundle contracts/silver/s_customers
contractforge plan contracts/silver/s_customers --adapter databricks
contractforge-databricks render contracts/silver/s_customers