Public CSV over HTTP
Uses http_file, explicit CSV options, source completeness and overwrite mode.
Examples
Practical ingestion patterns for common data sources.
Examples should demonstrate ContractForge capabilities without hiding recurring behavior behind notebook workarounds. If a pattern is broadly useful, prefer a declarative contract or a reusable template.
NASA EONET REST API
Uses rest_api raw response mode and transform.shape.parse_json to model nested arrays.
Azure Blob / ADLS files
Uses object storage paths, schemas, partitioned folders, recursive lookup and multiple file formats.
RDS/Aurora JDBC
Uses JDBC partitioning, pushdown, RDS IAM token generation and hash diff writes.
Generate a starter project
contractforge init --bundle --layer bronze --target-schema raw --table b_orders --mode scd0_append
contractforge validate-project contracts/Scenario catalog
| Scenario | Typical runtime | What it demonstrates |
|---|---|---|
| Public CSV over HTTP | Databricks serverless | http_file, explicit CSV options, overwrite, source metadata and a serverless-safe download path. |
| Azure Blob SAS files | Azure Databricks single-node classic | Direct object storage credentials, recursive folders, schemas, multiple file formats and cluster-level storage configuration. |
| Auto Loader on Blob/ADLS | Databricks | cloudFiles, checkpoints, available_now, child run metrics and stream aggregation. |
| Supabase/Postgres JDBC | Databricks | JDBC driver, pushdown, partitioned reads, watermarking, SCD1/hash diff patterns. |
| AWS S3 from Azure Databricks | Classic single-node | S3A credential setup, explicit region/session token behavior and cross-cloud object storage constraints. |
| RDS/Aurora IAM | Classic cluster | JDBC auth.type=rds_iam, botocore/default credential chain and network prerequisite diagnosis. |
Template families
Templates provide opinionated starting points for common connector, transform and write-mode combinations.
contractforge templates list
contractforge templates wizard --layer silver --source jdbc --mode scd1_hash_diff
contractforge templates write jdbc_hash_diff --output contracts/silver/s_ordersSmall files and partitioned object storage
Use explicit schemas, recursive lookup and optional regex filters for advanced file selection.
REST raw payloads
Keep API retrieval in the connector and model nested payloads through transform.shape.
JDBC incremental
Combine watermark, pushdown, partitioned reads, quality gates and deterministic merge semantics.
Auto Loader available-now
Use stream control tables to verify micro-batch counts and row totals.
Example design rule
Examples are not allowed to hide framework gaps with ad-hoc Spark code. If a real source requires common behavior, prefer improving the library or documenting a reusable pattern.
Design principle
Reusable source behavior belongs in connectors, reusable shaping belongs in transform.shape, and source-specific business logic belongs in project code or downstream transformations.