Skip to main content

Snowflake Capability And Evidence Parity

Purpose

This spec defines how a future contractforge-snowflake adapter should map ContractForge Core contract parameters and evidence/control-table fields to Snowflake-native capabilities.

The goal is not to make the core Snowflake-aware. The goal is to give the Snowflake adapter a researched implementation target:

ContractForge contract semantics
-> Snowflake capability declaration
-> Snowflake SQL / task / stream / policy implementation
-> canonical ContractForge evidence tables populated with Snowflake values

The sources for this document are Snowflake's official documentation. The adapter must verify behavior in a real Snowflake account before changing any REVIEW_REQUIRED item to SUPPORTED.

Reference Snowflake Surfaces

Snowflake surfaceContractForge use
SQL warehousePrimary runtime for table, SQL, staged file and merge-based ingestion.
Database and schemaMapping for target.catalog and target.schema.
Tables, views and stagesMapping for governed table sources and staged file sources.
COPY INTO <table>Bounded staged file loading and load validation.
SnowpipeContinuous or near-continuous staged file loading when the contract asks for file discovery beyond simple batch loads.
StreamsChange-data capture over Snowflake tables and views.
TasksScheduled execution, task graphs and stream-triggered jobs.
Row access policiesRow filter implementation.
Masking policiesColumn mask implementation.
Object tags and commentsAnnotation and governance metadata implementation.
Account Usage and Information SchemaNative evidence source for queries, tasks, copy loads, costs, policies, tags and object metadata.
Data Metric FunctionsOptional native quality metric surface.

Primary references:

Adapter Subtargets

The Snowflake adapter should not declare one vague snowflake capability. It should declare explicit subtargets.

SubtargetStatus targetPurpose
snowflake_sql_warehousePrimarySQL execution over warehouse compute. Handles table, SQL, file-stage, quality, merge and evidence writes.
snowflake_task_graphPrimary deployment pathRenders task/task-graph deployment artifacts for scheduled project execution.
snowflake_snowpipeReview-firstHandles staged file continuous loading where Snowpipe semantics preserve the contract.
snowflake_streams_tasksReview-firstHandles table CDC and incremental table-to-table workflows.
snowflake_snowparkFutureHandles complex transforms or connector behavior that is not clean in SQL alone.
snowflake_native_passthroughFutureDelegates to Snowflake-native connectors or marketplace/native app ingestion.

Core Parameter Parity

Sources

ContractForge parameterSnowflake mappingStatusNotes
source.type: tableSELECT ... FROM <database>.<schema>.<table>SUPPORTEDAdapter resolves logical refs to fully-qualified names.
source.type: viewSELECT ... FROM <database>.<schema>.<view>SUPPORTEDSecure view behavior must be recorded in planning details when relevant.
source.type: sqlSnowflake SQL querySUPPORTED_WITH_WARNINGSSQL dialect is Snowflake-specific. Portable {{ table_ref:* }} placeholders remain adapter-resolved.
source.type: staged_files with format: csv/json/parquetNamed internal/external stage plus named file format or stage default file format, read through a staged querySUPPORTED_WITH_WARNINGSCSV positional projection, JSON VARIANT payload/projection and Parquet projection are implemented. Other formats remain review-required.
source.type: csv/json/parquet/orc/avro/xml/textNormalize to staged_files or reviewed source-specific shorthandREVIEW_REQUIREDThe adapter accepts explicit staged_files first so file-format, stage and evidence boundaries are visible.
source.type: object_storageExternal stage over S3, Azure or GCSSUPPORTED_WITH_WARNINGSStorage integration, stage ownership and path privileges are environment-owned.
source.type: incremental_filesCOPY_HISTORY, Snowpipe, directory table, or adapter state tableREVIEW_REQUIREDSnowflake offers related surfaces, but exactly-once, ordering and reprocessing semantics depend on the chosen load path.
source.type: deltaExternal table/integration or imported stage patternREVIEW_REQUIREDDelta semantics are not Snowflake-native table semantics.
source.type: iceberg_tableSnowflake-managed or external Apache Iceberg supportSUPPORTED_WITH_WARNINGSCatalog, write support and metadata refresh behavior must be declared by the adapter.
source.type: http_file/http_csv/http_json/http_textExternal access integration, Snowpark procedure, or pre-stage file pathREVIEW_REQUIREDThe adapter should prefer pre-staged files for first implementation.
source.type: jdbc/postgres/mysql/sqlserver/oracle/redshift/db2/mariadbExternal function/procedure, Snowpark, or native connector patternREVIEW_REQUIREDSnowflake is not a general JDBC extraction runtime by default.
source.type: kafka_boundedSnowpipe Streaming, connector-managed stage, or external stream bridgeREVIEW_REQUIREDKafka replay semantics differ from ContractForge bounded Spark-style replay.
source.type: eventhubs_boundedExternal bridge to Snowflake stage/Snowpipe StreamingREVIEW_REQUIREDCross-cloud event offset semantics require design review.
source.type: delta_shareMarketplace/native sharing/external table patternSUPPORTED_WITH_WARNINGSNeeds adapter-specific source resolver.
source.type: native_passthroughSnowflake connector/native app/marketplace sourceSUPPORTED_WITH_WARNINGSMust return explicit warnings about connector-owned semantics and evidence gaps.
source.connection_pathCore-resolved shared connection before adapter planningSUPPORTEDSnowflake adapter receives the resolved source; it must not re-read arbitrary local paths.
source.authSnowflake secrets, integrations, key-pair auth or environment secret resolverREVIEW_REQUIREDThe adapter owns credential resolution and must redact evidence.
source.watermarkSQL predicate, stream offset, COPY_HISTORY marker or state table valueSUPPORTED_WITH_WARNINGSThe implementation path changes the strength of guarantees.
source.progress_locationContractForge state table or stage/copy markerREVIEW_REQUIREDThe core field is portable, but Snowflake implementation must choose table state vs native metadata.
source.schema_tracking_locationInformation Schema snapshot or adapter state tableREVIEW_REQUIREDSnowflake does not need a Databricks-style schema location; the adapter should map the intent to schema evidence.

Target And Write Semantics

ContractForge parameterSnowflake mappingStatusNotes
target.catalogDatabaseSUPPORTEDLogical catalog names should resolve through environment defaults.
target.catalog_typedatabase / metastore classificationSUPPORTEDUsed for planning and diagnostics; not a Snowflake DDL parameter.
target.schemaSchemaSUPPORTEDCreate/validate behavior is adapter-owned.
target.tableTableSUPPORTEDMust be quoted safely by Snowflake identifier rules.
layerNaming, comments, tags and evidenceSUPPORTEDLayer is ContractForge metadata, not a Snowflake primitive.
mode: appendINSERT INTO ... SELECT or COPY INTO appendSUPPORTEDFile loads should persist COPY_HISTORY details.
mode: overwriteTransactional replace pattern, CREATE OR REPLACE TABLE AS SELECT, or truncate/insertSUPPORTED_WITH_WARNINGSAtomicity and grants retention depend on the chosen pattern.
mode: upsertMERGE INTOSUPPORTEDSnowflake MERGE supports matched and not-matched clauses. Duplicate-source behavior must be prevalidated.
mode: hash_diff_upsertHash projection plus MERGE only for changed rowsSUPPORTED_WITH_WARNINGSHash function, null handling, collation and type casting must be adapter-defined and tested.
mode: historicalSQL MERGE/transaction pattern, possibly with streams/tasksREVIEW_REQUIREDLate-arriving records, effective dating and delete handling are project semantics, not a single Snowflake primitive.
mode: snapshot_reconcile_soft_deleteMERGE with matched/not matched source designSUPPORTED_WITH_WARNINGSRequires explicit snapshot boundary and source completeness proof.
schema_policy: strictCompare incoming schema to INFORMATION_SCHEMA.COLUMNS and fail driftSUPPORTEDStrict mode does not mutate target schema.
schema_policy: additive_onlyALTER TABLE ADD COLUMN after nullable additive diffSUPPORTED_WITH_WARNINGSType mapping and default/nullability decisions must be explicit.
schema_policy: permissiveReviewed DDL planREVIEW_REQUIREDType widening and incompatible changes require review.

Quality, Shape And Transform

ContractForge parameterSnowflake mappingStatusNotes
quality_rules.not_nullSQL COUNT_IF(column IS NULL) or DMF NULL_COUNTSUPPORTEDRuntime SQL is the first implementation; DMFs are optional governance integration.
quality_rules.required_columnsINFORMATION_SCHEMA.COLUMNS lookupSUPPORTEDMissing columns fail before write.
quality_rules.unique_keySQL grouped duplicate count or DMF DUPLICATE_COUNTSUPPORTEDComposite keys must use SQL grouping unless DMF fit is verified.
quality_rules.accepted_valuesSQL NOT IN or DMF ACCEPTED_VALUESSUPPORTED_WITH_WARNINGSLarge value sets should use reference tables instead of inline lists.
quality_rules.min_rowsSQL row count or DMF ROW_COUNTSUPPORTEDThe result goes to ctrl_ingestion_quality.
quality_rules.max_null_ratioSQL null count divided by row countSUPPORTEDDMF use is optional.
quality_rules.expressionsSnowflake SQL boolean predicateSUPPORTED_WITH_WARNINGSExpressions are dialect-specific and must be marked as such in planning.
on_quality_fail: failRaise error before target commitSUPPORTEDFailure must still persist error/run evidence where possible.
on_quality_fail: warnPersist quality evidence and continueSUPPORTEDRun status remains independent from quality status.
on_quality_fail: quarantineWrite failed rows to ctrl_ingestion_quarantine, then write passed rowsSUPPORTED_WITH_WARNINGSRow-level quarantine is supported for row-identifiable SQL rules; aggregate rules become recorded warnings.
shapeSQL over VARIANT, FLATTEN, casts, or SnowparkREVIEW_REQUIREDNested JSON and array explosion can be mapped, but parity with Spark/Glue shape behavior must be tested.
transform.castSnowflake CAST / TRY_CAST depending strictnessSUPPORTED_WITH_WARNINGSError behavior must match the contract.
transform.deriveSQL expression projectionSUPPORTED_WITH_WARNINGSDialect review required.
transform.standardizeSQL string functionsSUPPORTEDTrim, upper/lower and whitespace normalization are portable enough.
transform.deduplicateQUALIFY ROW_NUMBER() OVER (...) = 1SUPPORTED_WITH_WARNINGSDeterministic ordering is required.

Execution, Project And Deployment

ContractForge parameterSnowflake mappingStatusNotes
project.schedule.cronCREATE TASK ... SCHEDULE = 'USING CRON <expr> <timezone>'SUPPORTEDSnowflake task cron includes timezone in the schedule string.
project.schedule.timezoneTask cron timezoneSUPPORTEDTimezone behavior is task-specific and does not follow later session timezone changes.
project.depends_onTask graph AFTER <task>SUPPORTED_WITH_WARNINGSTask graph skip/failure propagation must be captured in deployment docs.
execution.freshnessTask schedule, stream-triggered task, Snowpipe, or external schedulerSUPPORTED_WITH_WARNINGSFreshness is intent; enforcement depends on runtime path.
execution.latency_targetPreserved in plan/evidence and optionally task/Snowpipe designSUPPORTEDIt is not a hard runtime guarantee by itself.
execution.preferredScheduled, task graph, stream-triggered task, Snowpipe or external orchestrationREVIEW_REQUIREDThe adapter must block if the preferred mode cannot preserve semantics and no fallback is declared.
execution.fallbackAlternative implementation pathSUPPORTEDThe planner may use the fallback only when explicitly allowed.
environment.artifacts.uriStage path, local output, Git artifact path, or object storage URISUPPORTED_WITH_WARNINGSThe Snowflake adapter should define accepted artifact schemes.
environment.evidenceSnowflake evidence database/schemaSUPPORTEDEvidence table names remain canonical ctrl_ingestion_*.
environment.parameters.snowflake.warehouseWarehouse bindingPLATFORM_EXTENSIONAdapter-owned deployment/runtime setting.
environment.parameters.snowflake.roleRole bindingPLATFORM_EXTENSIONAdapter-owned security setting.
environment.parameters.snowflake.task_schemaTask deployment namespacePLATFORM_EXTENSIONKeep out of ingestion contracts.

Annotations And Access

ContractForge parameterSnowflake mappingStatusNotes
annotations.table.descriptionCOMMENT ON TABLESUPPORTEDEvidence should record applied SQL/query id.
annotations.columns.*.descriptionCOMMENT ON COLUMNSUPPORTEDEvidence should record per-column status.
annotations.*.tagsSnowflake object tagsSUPPORTED_WITH_WARNINGSTag creation, namespace and allowed values are environment/governance decisions.
annotations.*.piiTags and optional masking policy recommendationSUPPORTED_WITH_WARNINGSPII tags do not automatically enforce masking unless access contract declares it.
access.grantsGRANT statements to roles/usersSUPPORTED_WITH_WARNINGSPrivilege vocabulary needs adapter mapping. Role hierarchy may create drift.
access.row_filtersRow access policiesSUPPORTED_WITH_WARNINGSSnowflake policies return boolean and are applied to tables/views. Principal-specific logic usually lives inside the policy expression.
access.column_masksMasking policiesSUPPORTED_WITH_WARNINGSPolicy signature/type compatibility must be checked. Some policy combinations have platform limits.
access.access_policy.on_driftCompare declared grants/policies with actual stateSUPPORTED_WITH_WARNINGSReconcile/revoke paths require explicit review due to role inheritance.
access.revoke_unmanagedRevocation planREVIEW_REQUIREDNever default to destructive access changes.

Evidence Storage Strategy

The Snowflake adapter should persist evidence in Snowflake tables with the same canonical names and columns defined by contractforge_core.evidence. These are ContractForge evidence tables, not Snowflake Account Usage views. Snowflake native views supply values; they do not replace the ContractForge evidence model.

Recommended default:

environment:
adapter: snowflake
evidence:
database: CONTRACTFORGE
schema: CF_EVIDENCE

Control tables should be append-only except lock/state helper surfaces where the adapter has a clearly documented concurrency strategy.

Control Table Mapping

ctrl_ingestion_runs

ContractForge field groupSnowflake source
Run identityContractForge run_id; Snowflake QUERY_HISTORY.query_id; task runs can also record TASK_HISTORY.graph_run_group_id.
Job identityTask database/schema/name from TASK_HISTORY; manual runs use procedure/query entrypoint.
TimingQUERY_HISTORY.start_time, QUERY_HISTORY.end_time, total_elapsed_time; task schedules use TASK_HISTORY.scheduled_time and completed_time.
StatusQUERY_HISTORY.execution_status plus adapter exception status; task status from TASK_HISTORY.state.
ErrorQUERY_HISTORY.error_code and error_message; adapter exception details after redaction.
Row metricsQUERY_HISTORY.rows_inserted, rows_updated, rows_deleted, rows_unloaded, rows_produced; adapter-computed rows_read, rows_written, rows_quarantined when native counters are insufficient.
Bytes/partition metricsbytes_scanned, bytes_written, bytes_written_to_result, partitions_scanned, partitions_total, spill/network fields into operation_metrics_json.
RuntimeWarehouse name/size, role/user, query tag, entrypoint and CURRENT_VERSION() where collected.
Table versionSnowflake has no Delta/Iceberg-style target table version for standard tables; keep table_version_before and table_version_after null and store query/transaction markers in operation_metrics_json.

ctrl_ingestion_state

ContractForge fieldSnowflake source
watermark_column, watermark_valueContract watermark and post-read high-watermark query result.
last_success_at_utc, last_status, last_run_idContractForge run finalization.
last_rows_writtenAdapter row metrics.
last_table_versionNull for standard tables; stream offset/copy marker may be stored in JSON.
last_watermark_candidateCandidate max watermark before commit.
state_details_jsonStream name/offset marker, COPY_HISTORY file marker, Snowpipe pipe name or task metadata.

Snowflake Account Usage views can have latency. State that affects correctness should be written by the adapter during the run instead of relying only on Account Usage reconciliation.

ctrl_ingestion_quality

ContractForge fieldSnowflake source
rule_nameContract rule name or native DMF name.
status, severity, failed_countAdapter SQL evaluation result.
observed_valueCount/ratio/value from SQL check or Data Metric Function.
details_jsonSQL predicate, DMF name, query id, sample limits and dialect classification.

Snowflake Data Metric Functions are useful evidence inputs, but they are not a complete replacement for ContractForge runtime quality checks because ContractForge must support fail/warn/quarantine behavior during ingestion.

ctrl_ingestion_quarantine

ContractForge fieldSnowflake source
record_payloadOBJECT_CONSTRUCT(*) / TO_JSON payload from failed row projection.
record_refOptional staging table key, target natural key or file/row reference.
reason, rule_nameFailed ContractForge row-level rule.
quarantined_at_utcAdapter timestamp at quarantine write.

Aggregate quality failures cannot produce row-level quarantine without an explicit row predicate. The adapter should record those as warnings or failures, not invent quarantined rows.

ctrl_ingestion_errors

ContractForge fieldSnowflake source
error_type, error_classAdapter exception classifier plus Snowflake error category when available.
error_messageRedacted adapter error plus QUERY_HISTORY.error_message.
stack_traceProcedure/Snowpark stack when available; otherwise null.
query_id / runtime_context_jsonQUERY_HISTORY.query_id, role, warehouse, task metadata.

ctrl_ingestion_schema_changes

ContractForge fieldSnowflake source
Before/after schemaAdapter schema snapshots from INFORMATION_SCHEMA.COLUMNS.
Applied DDLALTER TABLE / CREATE TABLE statement and QUERY_HISTORY.query_id.
Drift classificationCore schema policy result: strict/additive/permissive.

ctrl_ingestion_streams

ContractForge fieldSnowflake source
Batch identityTask run id, graph run group id, stream name, Snowpipe pipe name or copy batch marker.
Source progressStream offset state, COPY_HISTORY.file_name, last_load_time, pipe received time.
MetricsRows loaded, parsed rows, error count, file size, task/query metrics.
StatusTask/COPY/Snowpipe status normalized to ContractForge status.

Streaming and available-now semantics must remain REVIEW_REQUIRED until the adapter proves checkpoint/state behavior for each source family.

ctrl_ingestion_lineage

ContractForge fieldSnowflake source
Input/output objectsACCESS_HISTORY direct/base/modified objects where available.
Query identityquery_id, query tag and execution timestamp.
Event payloadNative ACCESS_HISTORY object arrays plus ContractForge source/target refs.

ACCESS_HISTORY is an audit source; the adapter should still emit a ContractForge lineage event during the run so dashboards do not depend only on Account Usage latency.

Runtime execution now writes an immediate ContractForge/OpenLineage-style COMPLETE event with the contract run id, source reference, target table, row count and Snowflake query ids. Native ACCESS_HISTORY reconciliation remains a delayed Account Usage path: it probes by structured QUERY_TAG run_id and returns PENDING without writing rows until matching native lineage rows are published.

ctrl_ingestion_annotations

ContractForge fieldSnowflake source
Table/column descriptionsApplied COMMENT statements and query ids.
TagsTAG_REFERENCES table function or Account Usage view, plus applied SQL.
StatusApplied/validated/warned/failed per annotation.

ctrl_ingestion_access

ContractForge fieldSnowflake source
GrantsApplied GRANT statements and information schema/account usage checks.
Row access policiesCreated/applied row access policy DDL and POLICY_REFERENCES.
Masking policiesCreated/applied masking policy DDL and POLICY_REFERENCES.
Drift statusAdapter comparison of declared access contract and current Snowflake state.

ctrl_ingestion_operations

Operations fields come from ContractForge operations.yaml. Snowflake can add task name, schedule, warehouse, alert integration and query tag evidence, but it must not redefine ownership, criticality or SLA semantics.

ctrl_ingestion_explain

ContractForge fieldSnowflake source
plan_textEXPLAIN output or profile/report reference.
explain_formatTEXT or JSON when supported by the rendered statement.
captured_at_utcAdapter timestamp.
details_jsonQuery id/profile URL when available.

Runtime execution captures EXPLAIN USING TEXT for the rendered write statement by default and stores failures as run metrics without failing the ingestion run. Contracts can disable capture with extensions.snowflake.explain_enabled: false.

ctrl_ingestion_cost

ContractForge fieldSnowflake source
signal_namequery_count, bytes_scanned, execution_time_ms, cloud_services_credits, rows_produced, warehouse_count, attributed_compute_credits, query_acceleration_credits, later snowpipe_bytes_billed.
signal_valueQUERY_HISTORY totals, QUERY_ATTRIBUTION_HISTORY.credits_attributed_compute, warehouse metadata counts, or later COPY_HISTORY.bytes_billed for Snowpipe.
payload_jsonWarehouse id/name, query id, root query id, query tag, cloud-services credits, acceleration credits, bytes scanned/written and task metadata.

Cost is an operational signal, not a billing reconciliation. Account Usage latency means cost evidence may be appended by a later reconciliation command. If Account Usage has not produced matching rows yet, reconciliation reports a pending status without inserting duplicate evidence. The same pending path is used when the executing role cannot read Account Usage, with a warning so operators can grant the needed Snowflake database role separately from runtime ingestion privileges.

ctrl_ingestion_metadata

The adapter fills ContractForge component metadata, core version, adapter version, control schema version, Snowflake account/region when configured, warehouse/role defaults and runtime feature flags. Sensitive values must be redacted.

Query Tag Requirement

The Snowflake adapter should set a structured QUERY_TAG for every statement it owns. At minimum:

{
"product": "contractforge",
"adapter": "snowflake",
"run_id": "<contractforge-run-id>",
"project": "<project-name>",
"target": "<database.schema.table>"
}

This gives QUERY_HISTORY, QUERY_ATTRIBUTION_HISTORY, ACCESS_HISTORY and task diagnostics a stable join key back to ContractForge evidence.

Review Boundaries

The initial Snowflake planner should return REVIEW_REQUIRED or SUPPORTED_WITH_WARNINGS for these areas until proven by real tests:

  • incremental_files using Snowpipe or copy history.
  • Kafka/Event Hubs bounded replay.
  • historical with late-arriving rows, deletes and overlapping validity windows.
  • Snapshot soft delete without a proven complete source snapshot.
  • Row access and masking policies with principal-specific logic.
  • Tag-based masking policy generation.
  • Complex shape operations over nested VARIANT payloads.
  • Quarantine for aggregate quality rules.
  • Type widening and incompatible schema changes.
  • Cost attribution when Account Usage latency delays query-cost rows.

Initial Capability Declaration Target

The first Snowflake adapter capability set should be conservative:

AreaInitial status
Table/view/sql sourcesSUPPORTED
Staged file batch loadSUPPORTED_WITH_WARNINGS
AppendSUPPORTED
OverwriteSUPPORTED_WITH_WARNINGS
current-state upsertSUPPORTED
hash-diff upsertSUPPORTED_WITH_WARNINGS
historicalREVIEW_REQUIRED
Snapshot soft deleteSUPPORTED_WITH_WARNINGS
Strict schema policySUPPORTED
Additive schema policySUPPORTED_WITH_WARNINGS
Core quality rulesSUPPORTED
Quality quarantineSUPPORTED_WITH_WARNINGS
Annotations commentsSUPPORTED
TagsSUPPORTED_WITH_WARNINGS
GrantsSUPPORTED_WITH_WARNINGS
Row access policiesSUPPORTED_WITH_WARNINGS
Masking policiesSUPPORTED_WITH_WARNINGS
Scheduled projectsSUPPORTED
Task dependenciesSUPPORTED_WITH_WARNINGS
Cost evidenceSUPPORTED_WITH_WARNINGS
Lineage evidenceSUPPORTED_WITH_WARNINGS

Acceptance Tests For A Future Adapter

  1. Plan the canonical Supabase medallion project against Snowflake and produce exact warnings/review items.
  2. Render SQL for table, SQL and staged file sources without importing a Snowflake client in the core.
  3. Execute append, overwrite and current-state upsert in a real Snowflake account.
  4. Verify ctrl_ingestion_runs, state, quality, quarantine, errors, schema_changes, annotations, access, lineage, operations, explain and cost tables use the core schema.
  5. Join ContractForge run_id to Snowflake QUERY_HISTORY through QUERY_TAG.
  6. Capture task evidence for a scheduled project with at least one dependency.
  7. Apply comments/tags and validate them through TAG_REFERENCES.
  8. Apply row access and masking policies and validate them through POLICY_REFERENCES.
  9. Prove that historical remains REVIEW_REQUIRED until the project declares the late-arriving, delete and effective-date policies needed for safe execution.