Table and SQL

stable

Use catalog-native sources when data is already registered in Spark or Unity Catalog.

When to use it

Choose table, delta_table or view for registered objects. Choose sql when the source must be a reviewed extraction query rather than a reusable catalog object.

Registered tables

Catalog-native sources

Use this for Unity Catalog, Hive metastore or Spark catalog objects already available to the runtime.

Views

Shared source logic

Prefer views when source-side filtering or joins are reused by more than one ingestion contract.

SQL extracts

Reviewed predicates

Use SQL for bounded extraction logic. Promote complex business transformations to views, notebooks or Silver/Gold contracts.

Internal lakehouse

Fastest path

This connector has no external driver or credential layer beyond catalog permissions.

Runtime requirements

Requirement	Details
Catalog access	The job principal must be able to read the source table, view or SQL dependencies.
Fully qualified names	Use catalog/schema/table names for external tables. Use logical refs for tables produced by other ContractForge contracts.
SQL review	Keep inline SQL small enough to review. Large logic should live in versioned views or transformations.
Watermark columns	If using incremental reads, the selected table/query must expose the watermark columns.

Basic example

source:
  type: connector
  connector: table
  table: main.raw.orders

source:
  type: connector
  connector: sql
  query: |
    SELECT id, updated_at, amount
    FROM main.raw.orders
    WHERE status = 'closed'

Logical table refs

Use logical refs when the source is another table managed by ContractForge. This keeps medallion contracts portable across Databricks, AWS and future adapters.

source:
  type: table
  ref: bronze.b_products_jdbc

Inline SQL can use the same neutral reference:

source:
  type: sql
  query: |
    SELECT *
    FROM {{ table_ref:silver.s_product_tags }}

The core validates the layer.table reference. Databricks resolves it to a Unity Catalog-style name; AWS resolves it to Glue Catalog/Iceberg.

Supported source types

Connector	Use
table	Registered runtime table, or logical source.ref to a table produced by another ContractForge contract.
delta_table	Registered Delta table when you want to make the storage type explicit.
view	Registered view used as source object.
sql	Inline SQL query declared in the contract; may include {{ table_ref:layer.table }} placeholders.

Advanced patterns

Use SQL sources for small, reviewed extraction predicates. If the SQL becomes a business transformation, promote it to a view, notebook step or a Silver/Gold contract.

source:
  type: connector
  connector: sql
  query: |
    SELECT order_id, updated_at, amount
    FROM main.raw.orders
    WHERE updated_at >= timestamp '2026-01-01'
watermark_columns: [updated_at]

Operational validation

SELECT run_id, status, source_connector, source_table, source_query, rows_read, rows_written
FROM main.ops.ctrl_ingestion_runs
WHERE source_connector IN ('table', 'delta_table', 'view', 'sql')
ORDER BY started_at_utc DESC;

Common issues

Symptom	Likely cause	Action
Table not found	Unqualified name, wrong catalog/schema context or logical ref naming that cannot be inferred.	Use fully qualified names for external sources; use source.ref or explicit table_ref for ContractForge-managed tables.
Permission denied	Job principal lacks catalog/table grants.	Grant source read permissions outside ContractForge.
Inline SQL is hard to review	Business transformation is hidden in source extraction.	Move logic to a view or downstream contract.

How this connector fits the contract

Keep extraction concerns in source, structural normalization in transform, validation in quality_rules and target semantics in mode. This separation keeps examples portable and prevents connector-specific workarounds from becoming hidden business logic.