# dbt — Deep dive: Every Line Has a Source: Inside the dbt Contract Generator CoreModels publishes governed meaning *into* a dbt project. You call Generate and get back dbt model property files with enforced contracts — one per model, colocated with the model's own `.sql` — which your own pull request flow lands in the repo. CoreModels never writes to your repo, never opens the PR, never runs dbt, and never connects to a warehouse; it returns files. Which means the files have to be checkable. Every line the generator emits traces back to a fact recorded in the governed model, and where a fact is missing it omits the line and says why rather than guessing. This is the mechanism in the order it runs, so an engineer reading the output can verify it against the rules instead of trusting it. # Every Line Has a Source: Inside the dbt Contract Generator CoreModels publishes governed meaning *into* a dbt project. You call Generate and get back dbt model property files with enforced contracts — one per model, colocated with the model's own `.sql` — which your own pull request flow lands in the repo. CoreModels never writes to your repo, never opens the PR, never runs dbt, and never connects to a warehouse; it returns files. Which means the files have to be checkable. Every line the generator emits traces back to a fact recorded in the governed model, and where a fact is missing it omits the line and says why rather than guessing. This is the mechanism in the order it runs, so an engineer reading the output can verify it against the rules instead of trusting it. ## Step 1 — Selecting the models The generator walks the governed schema's types in order and applies four filters, in this order: 1. **Resource kind.** A type whose recorded dbt resource kind is anything other than `model` is skipped silently. Seeds, sources, and snapshots import as governed types and are governed like everything else, but they never become model contracts — a `models:` block is for models. A type with *no* recorded resource kind is kept: that is a governed-first entity with no dbt origin, and publishing it into the project is the whole point. 2. **Ephemeral.** A model materialized `ephemeral` is skipped with a lossiness record. An ephemeral model is a CTE inlined into its consumers; there is no relation in the warehouse for dbt to enforce a contract against. 3. **The name filter.** When the request carries `typeNames`, a model survives only if the list contains its dbt model name or its governed label. This filter runs *after* the ephemeral check, so the ledger can carry an ephemeral-skip record for a model you did not ask for — an artifact of ordering, not a statement about your selection. 4. **Columns.** A governed type with no resolvable elements is skipped with a lossiness record. dbt rejects a contract-enforced model with no columns, and a single such block would fail the parse of the entire file. If nothing survives, Generate **fails** — `No eligible models found to generate contracts for.` — rather than returning a file with an empty `models:` list. An empty contract file is worse than an error, because it looks like an answer. The model's name comes from the unique id recorded at import: its last dot-segment, or the second-to-last when the last is a version suffix like `v2`, so the block is named for the model rather than for its versioned id. A governed-first type with no recorded unique id has its label snake-cased instead. ## Step 2 — Planning the files Colocation is the reason this generator was rebuilt: dbt repos keep properties next to the models they describe, and a single repo-wide file stops being reviewable somewhere around the third dozen model. The target path comes from `original_file_path`, captured at import into the dbt metadata mixin under the key `dbt.path` inside `__dbtMeta`. The generator takes the directory of that path — `models/staging/stg_orders.sql` yields `models/staging` — normalizing backslashes and trimming slashes on the way. When the path was never recorded, because the model predates path capture or because the type never came from dbt at all, the directory falls back to `models`. Layout is chosen by the `layout` option (REST `extra.layout`, MCP `layout`): | `layout` | File for a model at `models/staging/stg_orders.sql` | Fallback with no recorded path | |---|---|---| | `model` *(default)* | `models/staging/stg_orders.yml` | `models/stg_orders.yml` | | `folder` | `models/staging/_coremodels__models.yml` | `models/_coremodels__models.yml` | | `single` | `models/coremodels_contracts.yml` | `models/coremodels_contracts.yml` | An unrecognized layout value is not an error: the generator falls back to `model` and records a lossiness note naming the value it did not understand and the three it accepts. Each planned file becomes one returned artifact — `name` is the repo-relative path, `kind` is `yaml`, `content` is the file text — and files come back in the order their first model was encountered, models inside a file keeping the governed schema's order. ## Step 3 — The shape, in emission order ```yaml # Generated by CoreModels — governed model contracts. # Meaning changes belong in CoreModels; regenerate this file rather than editing it. version: 2 models: - name: stg_orders description: "One row per placed order, staged from the order service." config: contract: enforced: true materialized: view meta: coremodels: maps_to: - standard: "schema.org" uri: "https://schema.org/Order" columns: - name: order_id description: "Natural key issued by the order service. [schema.org: https://schema.org/orderNumber]" data_type: "varchar(64)" constraints: - type: not_null data_tests: - unique - not_null meta: coremodels: maps_to: - standard: "schema.org" uri: "https://schema.org/orderNumber" - name: order_status data_type: "varchar(20)" constraints: - type: not_null data_tests: - not_null - accepted_values: values: ["processing", "in_transit", "delivered", "cancelled"] meta: coremodels: vocabulary: "Order Status" vocabulary_maps_to: - standard: "schema.org" uri: "https://schema.org/OrderStatus" term_maps_to: - value: "cancelled" uri: "https://schema.org/OrderCancelled" - name: customer_id data_type: "varchar(64)" constraints: - type: not_null data_tests: - not_null - relationships: to: ref('stg_customers') field: customer_id ``` The order is fixed. Within a model: `name`, `description`, `config` (with `contract.enforced: true` first and `materialized` after it), `meta`, `columns`. Within a column: `name`, `description`, `data_type`, `constraints`, the tests key, `meta`. Optional keys are omitted entirely rather than emitted empty — no `description:` without a description, no `materialized:` without a recorded materialization, no `constraints:` without a not-null check, no tests key without at least one test, no `meta:` without a binding or a vocabulary. If you diff two generations and a key disappeared, a fact disappeared. ## Step 4 — Where each value comes from | Emitted | Source | When the source is missing | |---|---|---| | `description` (model, column) | the governed description recorded on the node | key omitted | | `materialized` | the materialization recorded at import | key omitted | | `contract.enforced: true` | always, on every emitted model | — | | `data_type` | the vendor-native type recorded at import | warehouse-neutral fallback | | `constraints: - type: not_null` | a recorded not-null check | key omitted | | `unique` / `not_null` tests | recorded unique and not-null checks | omitted | | `accepted_values` | the governed taxonomy's terms | omitted | | `relationships` | the governed reference plus its recorded target field | omitted, with a reason | **`data_type` is never blank.** Under an enforced contract dbt requires a type on every column, and a missing one fails at parse — which is also why the dbt audit rules include `contract-column-missing-type`. The generator prefers the exact native string recorded at import (`varchar(64)`, `numeric(38,2)`), because that is what the warehouse actually holds. For a governed-first column that never round-tripped from dbt, it falls back to a warehouse-neutral type derived from the governed primitive: `integer`, `float`, `boolean`, `timestamp`, or `text`. A column governed by a vocabulary falls back to `text` — its values are constrained by the `accepted_values` test, not by the column type — and so does a reference column, which carries the foreign key value. **Constraints and tests come from the same recorded checks and mean different things.** A not-null check produces both a `constraints` entry, which dbt turns into a platform constraint on the built relation, and a `not_null` data test, which runs at test time. A unique check produces the `unique` test only. Test order within a column is stable: `unique`, then `not_null`, then `accepted_values`, then `relationships`. **`accepted_values` comes from the governed vocabulary, not from the model.** When a column's value type is a governed taxonomy with at least one term, those terms become the `values` list. This is where publication earns its keep: one governed vocabulary generates the same `accepted_values` list in every model that uses that field, so the enum stops being a per-model copy that drifts. A taxonomy with no terms produces no test and no vocabulary metadata. **`relationships` is generated or omitted, never guessed.** The target model comes from the governed reference. The target field comes first from the recorded relationship check's detail — the `field` captured when the original test was imported — and second, when there is none, from the first column on the target type carrying a unique check, which is the natural key such a test points at. If neither exists, or the reference points at a type that is not in the governed schema, the test is omitted and a lossiness record says which of the two happened. A guessed literal `id` would compile fine and fail against a real warehouse; an omission with a reason is a task, not a false negative. ## Step 5 — The `meta.coremodels` block dbt has no first-class slot for an ontology binding, so governed `mapsTo` assertions ride out two ways. Structurally, under `meta.coremodels.maps_to` — a list of `{standard, uri}` pairs, emitted on the model and on each column that carries bindings. This always rides, and it is what lets a catalog, a downstream tool, or an agent resolve the column to the term an Admin actually bound instead of inferring meaning from the column name. Textually, appended to the column description as `[standard: uri]`, comma-joined when there are several. This is on by default and switched off with `extra.iriInDescription = "false"`. The reason it exists is narrow and practical: `description` is the only slot dbt's `persist_docs` carries into the warehouse column `COMMENT`, so it is the only path by which a bound term reaches someone querying the warehouse directly. A column governed by a vocabulary emits three more keys under the same block: `vocabulary`, the vocabulary's name; `vocabulary_maps_to`, when the vocabulary itself is bound; and `term_maps_to`, a list of `{value, uri}` for the terms that carry bindings — each contributing its first binding, and unbound terms simply not appearing. The `values` list in the test and the `term_maps_to` list in the metadata are therefore not always the same length, and that is expected. ## Step 6 — The lossiness ledger Generate returns a ledger alongside the artifacts. Nothing in it is an exception path; each record is a fact about the gap between what CoreModels governs and what dbt can carry. | Kind | Subject | What it means here | |---|---|---| | Structural drop | model name | **Ephemeral skip.** The model cannot carry a contract; nothing was emitted for it. | | Structural drop | model name | **No columns.** The governed type has no elements, and a contract needs at least one column. | | Structural drop | model name | **Duplicate patch.** The model already has properties in another file, named in the message. | | Structural drop | `model.column` | **Hierarchical vocabulary.** The governed vocabulary has parent and child terms; `accepted_values` is a flat list, so the hierarchy is not represented in the YAML. | | Semantic narrowing | `model.column` | **Relationships omitted.** Either the target type is not in the governed schema, or the target field is not recorded and the target has no unique-tested column. | | Semantic narrowing | `targetVersion` | **Legacy tests key**, or a version string that would not parse. | | Semantic narrowing | `layout` | **Unknown layout**, with the value received and the valid ones. | Two of these deserve more than a row. **Duplicate patches:** dbt refuses two property blocks for one model, so if the model's `patch_path` was recorded at import — stored as `dbt.patchPath` — and it names a file other than the one being generated, the generator does not touch that file. It records the conflict and names the file whose block must be removed. Overwriting would be the wrong move: that file may carry properties for models CoreModels does not govern, and deleting someone else's blocks to make room is not a migration anyone reviewed. Clearing them is a one-time step in the first publication PR. **The tests key:** by default the generator emits `data_tests:`, which dbt 1.8 and newer read. Passing a `targetVersion` of `1.7` — or any `1.x` below 1.8 — switches to the legacy `tests:` key and records a note saying that 1.8 and newer read `data_tests:`. A version string that will not parse stays on the modern key and records that it did. Enforced contracts themselves require dbt 1.5 or newer, so a target version below 1 gets the legacy key together with a note that contracts are unavailable there at all. ## Step 7 — Quoting, and why the output is byte-stable The YAML is emitted directly rather than through a serializer, because the shape is small and closed, and the quoting rule is deliberately conservative. A scalar is written plain only when it is non-empty, consists entirely of letters, digits, `_`, `-`, and `.`, does not begin with a digit, and is not one of `true`, `false`, `null`, `yes`, `no`, `on`, `off` in any case. Everything else is double-quoted, with `\\`, `\"`, `\n`, `\r`, `\t`, and `\uXXXX` escapes for other control characters — so a multi-line description cannot break the document's structure. Some values are always quoted regardless: `accepted_values` entries, every `standard` and `uri`, the vocabulary name, and each `term_maps_to` value. That is why a vocabulary containing `NO`, `Y`, or `1` survives the round trip instead of arriving at a YAML 1.1 parser as a boolean or an integer. It is also why `data_type: "varchar(64)"` is quoted — parentheses are not in the safe set — while `data_type: text` is not. One value bypasses the quoter on purpose: `to: ref('stg_customers')` is written literally, because that is the string dbt expects and it is already a legal plain scalar. Nothing in the output derives from the clock, the run, or the environment. There is no timestamp, no run id, no fingerprint, no incidental ordering: models follow the governed schema's type order, columns follow their type's element order, files follow first appearance. The same governed model with the same options produces byte-identical files, which is what makes regeneration safe as a routine step in a PR — a regeneration with no governance changes produces an empty diff, and any diff you do see is a meaning change someone made in CoreModels, reviewable line by line. That is also the honest boundary of the mechanism. Importing a manifest gives you an accurate description of your estate and very little else; contracts get their value from what a human adds on top — descriptions, vocabularies, ontology bindings — and this generator is the path by which that added meaning gets back into the project where dbt can enforce it. For the extraction, the calls, and a first publication end to end, see the dbt quickstart in the CoreModels integration docs.