Two Loops and an Empty Diff: dbt Contract Automation in CI

Try this on a branch first, because it reframes what the gate is for. Take a model with an accepted_values test, delete the test, and open a pull request.

Everything passes.

Of course it does — dbt tests what is in the branch, and a deleted test is simply one that no longer runs. Branch-versus-main tooling has nothing to flag either, because the thing that changed is the absence of a check. Every mechanism in the pipeline is comparing the PR to the previous commit, and none of them is comparing it to what the team agreed the column means. That is the gap this gate exists for: it is the only check in the pipeline whose reference point is the agreement rather than the last commit. A removed permitted-value set, a widened enum, a type that quietly changed or a dataset that vanished each come back as a named finding with a stable code.

Once CoreModels publishes contracts into a dbt project, two different things can go wrong, and automating them as one job is the mistake most teams make first. The repo can move away from the governed model — a column retyped, a governed vocabulary quietly trimmed. Or the governed model can move and the published files go stale: a description rewritten in governance, an ontology term bound, and the YAML in the repo still says what it said last month. Different triggers, different credentials, different verdicts. Two loops, then — an audit gate that fails pull requests, and a regeneration job that opens them.

Loop one: the audit gate

The gate calls the machine-to-machine audit route, POST https://coremodels.example.com/v1/{PROJECT_ID}/integrations/dbt/audit, which accepts CoreModels user API keys rather than login tokens. It runs read-only at Viewer role — the key you hand CI needs no write access at all — and the response comes back in the standard API envelope, so the report lives under data.*. The contract is one line: data.errorCount > 0 means the change violates governed meaning; fail the build.

The only required artifact is manifest.json, and dbt parse produces it without connecting to a warehouse. Adding catalog.json from dbt docs generate sharpens the type comparison but needs a real run, so most teams gate on the manifest alone.

steps:
  - name: compile the manifest
    run: |
      dbt deps
      dbt parse                    # no adapter connection required

  - name: audit against governed meaning
    env:
      CM_URL: $CM_URL              # CI variable
      CM_PROJECT: $CM_PROJECT      # CI variable: the governing project id
      CM_KEY: $CM_KEY              # CI secret: a Viewer-scoped user API key
    run: |
      set -euo pipefail
      jq -n --rawfile manifest target/manifest.json \
        '{artifacts: {manifest: $manifest}, recordHistory: true}' > audit-request.json
      curl -sS -o audit.json -X POST \
        "$CM_URL/v1/$CM_PROJECT/integrations/dbt/audit" \
        -H "Authorization: Bearer $CM_KEY" -H "Content-Type: application/json" \
        --data-binary @audit-request.json
      test "$(jq -r '.success' audit.json)" = "true"
      jq -r '.data.markdown' audit.json > audit-comment.md
      test "$(jq -r '.data.errorCount' audit.json)" -eq 0

Read the last three lines as three distinct checks in a deliberate order. .success must be true first: false means the audit could not run at all — an unparseable manifest, an unknown vendor — which is a broken pipeline, not a clean result, and it must fail loudly rather than gate on a number that was never computed. Then data.markdown is written out: it is a complete formatted report, verdict and counts on top, and it is meant to be posted verbatim as the PR comment body by whatever forge CLI your pipeline already has. Only then does the gate itself run.

What trips it: the dbt conformance findings contract-not-enforced (Error on public models, Info otherwise) and contract-column-missing-type (Error — an enforced contract with an untyped column fails dbt at parse), plus the drift findings from the shared audit layer, of which dataset-removed, field-type-drift, and enum-narrowed are Errors. enum-widened and enum-constraint-removed come back as Warnings — visible in every report, fatal only if you tighten the gate to warnings. key-column-untested is a Warning and source-no-freshness is an Info; both inform a build rather than stopping it.

The trail and the badge

recordHistory: true is opt-in bookkeeping — the audit verb is otherwise a pure query — and in CI you want it on. Each gated run lands in the project's rolling trail with trigger ci, its three counts, its per-code totals, and the artifact fingerprint, readable with a login token at GET graph/integrations/dbt/history/{PROJECT_ID}. The fingerprint is the quietly useful field: two runs with the same fingerprint audited a byte-identical manifest, so any change in findings between them came from the governed model moving, not the repo. The latest recorded run also renders as a shields-style SVG at GET v1/{PROJECT_ID}/integrations/dbt/badge on the same API-key surface — green for clean, yellow for warnings only, red for errors, gray when nothing has been recorded yet. Gray is informative rather than neutral: a project whose badge is gray has never recorded a run, which almost always means recordHistory was never turned on.

Loop two: regenerating the published contract

The second loop runs on a schedule or on demand, not on pull requests, because its trigger lives in governance rather than in the repo. It calls generate, writes the returned files into a branch, and opens a PR:

POST https://coremodels.example.com/graph/integrations/dbt/generate/{PROJECT_ID}
{ "extra": { "layout": "model" } }

Generate runs at Viewer role on the interactive surface — the v1 API-key surface carries audit and badge only — so this job authenticates as an interactive client, or drives the MCP tool generate_vendor_artifacts with vendor dbt and the same options. Plan the credentials accordingly; it is the one asymmetry between the loops. The response is artifacts, a list of {name, kind, content} where name is the repo-relative path, plus the lossiness ledger:

jq -r '.artifacts[] | @base64' generate.json > rows
while read -r row; do
  decode() { printf '%s' "$row" | base64 --decode | jq -r "$1"; }
  path="$(decode '.name')"
  mkdir -p "$(dirname "$path")"
  decode '.content' > "$path"
done < rows

Under the default model layout each file lands beside the model's own .sql — models/staging/stg_orders.sql gets models/staging/stg_orders.yml. Use folder for one file per model directory (_coremodels__models.yml), or single for one repo-wide file, kept for back-compat and unworkable past a few dozen models. What lands looks like this:

# Generated by CoreModels — governed model contracts.
# Meaning changes belong in CoreModels; regenerate this file rather than editing it.
version: 2

models:
  - name: stg_orders
    description: One row per order submitted through checkout.
    config:
      contract:
        enforced: true
      materialized: view
    columns:
      - name: order_id
        description: Surrogate key for the order. [schema.org: https://schema.org/orderNumber]
        data_type: varchar(36)
        constraints:
          - type: not_null
        data_tests:
          - unique
          - not_null
        meta:
          coremodels:
            maps_to:
              - standard: "schema.org"
                uri: "https://schema.org/orderNumber"
      - name: order_status
        data_type: varchar(20)
        data_tests:
          - accepted_values:
              values: ["placed", "shipped", "returned"]
        meta:
          coremodels:
            vocabulary: "Order Status"

Then the job commits the branch and opens the pull request through the team's own tooling. Say it plainly, because the boundary is the whole design: CoreModels returned bytes over HTTP. It does not hold a repo credential, does not open the PR, does not connect to your warehouse, and does not run dbt. Enforced contracts need dbt 1.5 or newer; the modern data_tests: key is emitted by default, and targetVersion of 1.7 (or any 1.x below 1.8) switches to the legacy tests: key and says so in the ledger, because dbt 1.8 and newer read data_tests:. Rolling out model by model is a typeNames list on the same call.

The empty diff is the assertion

Generation is deterministic: the same governed model and the same options produce byte-identical files. That turns the diff into a test.

git add -A models
if git diff --cached --quiet; then
  echo "published contracts already match governed meaning"
  exit 0            # nothing to review; no PR
fi

A nightly job that exits here costs nothing and tells you something true. When the diff is not empty, every line of it traces back to a governance edit somebody made — a description rewritten, a vocabulary term added, an ontology term bound — and the reviewer is reading a change in meaning rather than a change in formatting. Two things break the property on purpose: changing layout relocates every file, and changing targetVersion or iriInDescription rewrites every file. Pin those options in the job config and change them in their own pull request, or a churn diff of several hundred files will hide the one line that mattered.

The lossiness ledger is a CI signal

Generate fails outright when no models are eligible, but a successful run can still carry records that need a person. Block the branch on the duplicate-patch record: dbt refuses two property blocks for one model, so if a generated file lands while the model's old block still sits in another file, the next dbt parse fails. The generator never rewrites a file it does not own — that file may carry properties for models CoreModels does not govern — so it names the file whose block must be removed and leaves the migration to a human:

jq -r '.lossiness[] | [.kind, .path, .explanation] | @tsv' generate.json | tee lossiness.tsv
if grep -q "already has properties" lossiness.tsv; then
  echo "clear the duplicate property blocks named above before regenerating" >&2
  exit 1
fi

Warn and carry on for the rest. A SemanticNarrowing on a column means a relationships test was omitted because the target field is not recorded and the target model has no unique-tested column — the generator declines to guess a field name that would fail against a real warehouse. A StructuralDrop on a hierarchical vocabulary means accepted_values is a flat list and the parent/child structure did not survive. A SemanticNarrowing on targetVersion is the legacy-key note. Record and move on for skips: ephemeral models cannot carry contracts, and a governed type with no columns cannot become one. Both are expected in a healthy project, and both belong in the PR body so nobody wonders where a model went.

Where the loops meet

A regeneration PR is itself a dbt PR, so loop one audits it — exactly the check you want, since the contracts you just published have to survive the gate you already run. A clean pass typically clears contract-not-enforced and contract-column-missing-type in a single merge, because every emitted block sets enforced: true and every column carries a data_type. A failure is worth more than a pass: field-type-drift on a freshly generated contract means the model's SQL produces a different type from the one governance recorded, and no amount of YAML reconciles that. Someone has to decide which side is wrong.

What neither loop provides is worth stating so nobody automates against it. Generate emits model contract property files and nothing else — no sources.yml, no staging models, no project scaffold, which is ground dbt-codegen already occupies. There is no version history of the governed model, no diff between two points in time, and nothing that notifies you when a colleague edits meaning in CoreModels. That absence is precisely why loop two runs on a schedule: a nightly regeneration with an empty diff is the cheapest honest substitute for a change feed, and the day it is not empty, the diff says what changed.

The dbt quickstart that ships with the CoreModels integration docs carries the full route reference, the layout options, and runnable versions of both jobs.