Skip to main content

Snowflake parity

Snowflake adapter implementation has started with a planning/rendering-first package. The adapter preserves the same ContractForge contracts used by Databricks and AWS, then maps them to Snowflake-native SQL, warehouses, tasks, streams, policies and audit tables.

The core will not import Snowflake clients. Snowflake-specific behavior belongs in contractforge-snowflake.

Target architecture

LayerSnowflake mapping
RuntimeSQL warehouse first; Snowpark only when SQL cannot preserve the contract cleanly.
DeploymentTasks and task graphs for project schedules and dependencies.
File ingestionStages plus COPY INTO; Snowpipe only after review of incremental semantics.
Table CDCStreams plus tasks where change-data semantics match the contract.
GovernanceRow access policies, masking policies, grants, comments and tags.
EvidenceCanonical ctrl_ingestion_* tables in a Snowflake evidence database/schema.

Contract parity

ContractForge areaSnowflake status target
Table, view, SQL and bounded REST API sourcesSUPPORTED in planning
Staged file batch ingestionSUPPORTED_WITH_WARNINGS
incremental_filesREVIEW_REQUIRED until Snowpipe/copy-history state behavior is proven.
AppendSUPPORTED in planning
OverwriteSUPPORTED_WITH_WARNINGS because replacement and grant-retention choices matter.
current-state upsertSUPPORTED in planning through MERGE.
hash-diff upsertSUPPORTED_WITH_WARNINGS because hash, null, type and collation semantics must be adapter-defined.
historicalREVIEW_REQUIRED until late-arriving, delete and effective-date policies are explicit.
Quality rulesSUPPORTED for SQL checks; Snowflake Data Metric Functions can enrich evidence.
QuarantineSUPPORTED_WITH_WARNINGS for row-level rules only.
Schedule and timezoneSUPPORTED through Snowflake task cron syntax.
Row filtersSUPPORTED_WITH_WARNINGS through row access policies.
Column masksSUPPORTED_WITH_WARNINGS through masking policies.
Cost evidenceSUPPORTED_WITH_WARNINGS through Account Usage views, subject to latency.

Control-table mapping

Snowflake native telemetry should populate the same ContractForge evidence schema, not replace it.

ContractForge tableSnowflake source
ctrl_ingestion_runsQUERY_HISTORY, TASK_HISTORY, adapter runtime metrics.
ctrl_ingestion_stateAdapter-written state, high-watermarks, stream/copy markers.
ctrl_ingestion_qualitySQL quality results and optional Data Metric Function results.
ctrl_ingestion_quarantineFailed row payloads from row-level quality predicates.
ctrl_ingestion_errorsRedacted adapter exceptions plus Snowflake query/task error fields.
ctrl_ingestion_schema_changesINFORMATION_SCHEMA.COLUMNS snapshots plus applied DDL query ids.
ctrl_ingestion_streamsStreams, tasks, Snowpipe and copy-history batch markers.
ctrl_ingestion_lineageACCESS_HISTORY and adapter-emitted ContractForge lineage events.
ctrl_ingestion_annotationsCOMMENT statements and tag references.
ctrl_ingestion_accessGrants, row access policies, masking policies and POLICY_REFERENCES.
ctrl_ingestion_costQUERY_ATTRIBUTION_HISTORY, warehouse metering and Snowpipe bytes billed.

The adapter should set a structured Snowflake QUERY_TAG on every statement it owns so Account Usage rows can be joined back to ContractForge run_id.

The USGS GeoJSON parity project now uses the same source.type: rest_api bronze contract on Databricks, AWS and Snowflake. Snowflake hosted-procedure execution requires an external access integration such as CF_USGS_REST_ACCESS that allows outbound HTTPS to earthquake.usgs.gov:443.

For unauthenticated REST sources, create the integration with Snowflake's no-secret keyword form:

USE ROLE ACCOUNTADMIN;

CREATE OR REPLACE NETWORK RULE CONTRACTFORGE_TEST_DB.PUBLIC.CF_USGS_REST_RULE
TYPE = HOST_PORT
MODE = EGRESS
VALUE_LIST = ('earthquake.usgs.gov:443');

CREATE OR REPLACE EXTERNAL ACCESS INTEGRATION CF_USGS_REST_ACCESS
ALLOWED_NETWORK_RULES = (CONTRACTFORGE_TEST_DB.PUBLIC.CF_USGS_REST_RULE)
ALLOWED_AUTHENTICATION_SECRETS = none
ENABLED = TRUE;

GRANT USAGE ON INTEGRATION CF_USGS_REST_ACCESS
TO ROLE CONTRACTFORGE_INGEST_ROLE;

Use ALLOWED_AUTHENTICATION_SECRETS = none, without parentheses. The form (none) can be parsed as a secret named PUBLIC.NONE.

{
"product": "contractforge",
"adapter": "snowflake",
"run_id": "<contractforge-run-id>",
"target": "<database.schema.table>"
}

Native sources researched

The Snowflake parity spec is based on official Snowflake documentation for:

  • QUERY_HISTORY, TASK_HISTORY, QUERY_ATTRIBUTION_HISTORY, COPY_HISTORY and ACCESS_HISTORY;
  • COPY INTO, Snowpipe, streams and tasks;
  • MERGE;
  • INFORMATION_SCHEMA.COLUMNS;
  • Data Metric Functions;
  • row access policies, masking policies, tags and policy references.

Read the full architecture contract in Snowflake capability and evidence parity.

The detailed implementation roadmap is in Snowflake adapter implementation plan.