# ODCS — Automation: The Contract Pipeline Is a Plan File: Automating ODCS With Replay Automation is where schema tooling usually stops being honest. A conversion that a human runs once gets its output eyeballed; the same conversion in a nightly job gets trusted. So the two properties that matter most for automating data contracts are not features, they are guarantees: CoreModels transforms are **deterministic** - the same contract in produces byte-identical output every run, which makes diffs a real review surface - and every mapping executes from a **plan** that comes back in the response as JSON you can commit, review, and replay. This article builds an ODCS pipeline on those guarantees: a published subset of an internal contract, regenerated on every change, with drift caught by the engine instead of by a consumer. # The Contract Pipeline Is a Plan File: Automating ODCS With Replay Automation is where schema tooling usually stops being honest. A conversion that a human runs once gets its output eyeballed; the same conversion in a nightly job gets trusted. So the two properties that matter most for automating data contracts are not features, they are guarantees: CoreModels transforms are **deterministic** - the same contract in produces byte-identical output every run, which makes diffs a real review surface - and every mapping executes from a **plan** that comes back in the response as JSON you can commit, review, and replay. This article builds an ODCS pipeline on those guarantees: a published subset of an internal contract, regenerated on every change, with drift caught by the engine instead of by a consumer. ## The job The internal contract, `orders.odcs.yaml` - six properties, of which `customer_ref` must never appear in the published edition: ```yaml apiVersion: v3.1.0 kind: DataContract id: urn:datacontract:sales:orders name: orders version: 1.2.0 status: active domain: sales description: purpose: Governed view of confirmed customer orders. usage: Analytics and settlement reporting. servers: - server: analytics-pg type: postgres host: db.internal.example port: 5432 schema: - name: orders physicalName: orders_v1 physicalType: table logicalType: object description: One row per confirmed order. properties: - name: order_id logicalType: string physicalType: uuid primaryKey: true primaryKeyPosition: 1 required: true unique: true - name: customer_ref logicalType: string required: true - name: order_total logicalType: number physicalType: numeric(12,2) required: true quality: - metric: nullValues mustBe: 0 - name: placed_at logicalType: timestamp required: true - name: item_count logicalType: integer - name: gift logicalType: boolean ``` ## The plan is a file you can write You can derive a plan by calling `schema/map` with a mapping strategy - but a plan is just JSON, and for a redaction this small the clearest artifact is one you author directly and keep in the repo. `publish.plan.json`: ```json { "operations": [ { "kind": "TypeMapping", "origin": "Inferred", "sourceTypeId": "orders", "targetTypeId": "orders", "targetLabel": "orders" }, { "kind": "ElementMapping", "origin": "Inferred", "sourceElementIds": ["ordersOrderId"], "targetElementIds": ["ordersOrderId"] }, { "kind": "ElementMapping", "origin": "Inferred", "sourceElementIds": ["ordersOrderTotal"], "targetElementIds": ["ordersOrderTotal"] }, { "kind": "ElementMapping", "origin": "Inferred", "sourceElementIds": ["ordersPlacedAt"], "targetElementIds": ["ordersPlacedAt"] }, { "kind": "ElementMapping", "origin": "Inferred", "sourceElementIds": ["ordersItemCount"], "targetElementIds": ["ordersItemCount"] }, { "kind": "ElementMapping", "origin": "Inferred", "sourceElementIds": ["ordersGift"], "targetElementIds": ["ordersGift"] }, { "kind": "Drop", "origin": "Explicit", "sourceNodeId": "ordersCustomerRef", "sourceKind": "Element", "reason": "customer_ref is not published to the analytics contract." } ] } ``` The ids are the engine's element ids - the object name plus the camelCased property name (`ordersOrderId` for `orders.order_id`). Every mapping is explicit; the one omission is a `Drop` operation carrying its reason *inside the artifact*. A reviewer of this file in a pull request sees what will happen to every property before it happens to any of them. That is the entire point: human judgment lands once, visibly, instead of on every run, invisibly. ## Replay is the pipeline `plan/execute` runs a stored plan - the plan travels **as a string**, hence `--rawfile`: ```bash jq -n --rawfile s orders.odcs.yaml --rawfile p publish.plan.json '{ sourceFormat: "odcs", sourceSchema: $s, plan: $p, targetFormat: "odcs" }' \ | curl -sS -X POST \ "https://coremodels.example.com/graph/transform/plan/execute/$PROJECT_ID" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ --data-binary @- > publish-result.json ``` The produced contract, `jq -r '.schema' publish-result.json`: ```yaml apiVersion: v3.1.0 kind: DataContract id: urn:datacontract:sales:orders name: orders version: 1.2.0 status: active domain: sales description: purpose: Governed view of confirmed customer orders. usage: Analytics and settlement reporting. schema: - name: orders logicalType: object physicalName: orders_v1 physicalType: table description: One row per confirmed order. properties: - name: order_id logicalType: string physicalType: uuid required: true unique: true primaryKey: true primaryKeyPosition: 1 - name: order_total logicalType: number physicalType: numeric(12,2) required: true quality: [{"metric":"nullValues","mustBe":0}] - name: placed_at logicalType: timestamp required: true - name: item_count logicalType: integer - name: gift logicalType: boolean servers: [{"server":"analytics-pg","type":"postgres","host":"db.internal.example","port":5432}] ``` No `customer_ref`; everything else intact - head, physical facts, the quality check, the `servers` block. Run it twice and the outputs are identical. The ledger carries the decode-side entries (`servers` and `quality` preserved verbatim, declared as `SemanticNarrowing`); the drop itself is not a surprise to look for in the ledger, because it is the second thing anyone reading the committed plan sees. Two artifacts, two jobs: the plan states intent, the ledger states cost. Replay stays safe because a stored plan earns no shortcut: every execution passes the same universal validation gate as a freshly derived plan. If someone renames `customer_ref` upstream, the next run fails with the operation index and the id that stopped resolving - a drift alarm delivered by the same call that does the work. And an *unmapped* property is never silently lost on inferred plans either: a source property no operation maps or drops produces `StructuralDrop` - "Element is a member of the mapped type but no operation maps or drops it" - with the type and element named. ## Choosing a strategy when you do derive plans `schema/map`'s `mapping.kind` (and the MCP tool's `mappingKind`) offers three authoring routes, and ODCS changes the calculus a little: - **`inferred`** - label/type matching against a target hint; `caseInsensitive` defaults `true`. It requires a hint (without one: `The inference resolver requires a target IR to match against.`), and for contract-to-contract narrowing - legacy shape onto governed shape - it is usually all you need. - **`explicit`** - an authored guide: `autoMatchByMapsTo`, `fieldMappings` (`{sourceElementIds, targetElementIds, transformName}`), `taxonomyDirectives`, `drops`. One ODCS-specific caveat: `autoMatchByMapsTo` aligns constructs that share a semantic IRI, and the ODCS decoder lifts no field into that channel (`authoritativeDefinitions` are preserved verbatim, not read as mapping IRIs) - so for ODCS sources, matching comes from labels and authored entries, not IRIs. - **`ai`** - a server-side Claude proposer drafts the plan; the identical gate accepts or rejects it, one repair attempt maximum. It needs a server-held Anthropic key and Editor/Admin membership, and it sends the schema content to the Anthropic API. Use it to draft a gnarly first mapping, then freeze the returned plan and never call the proposer in the pipeline. Whichever route authored it, the plan is what you commit. Strategies are for authoring; pipelines replay. ## The gate reads your guide before your pipeline does Three rejections worth meeting in development rather than in CI, each arriving in the envelope with a path: - A typo'd guide key is refused, not skipped: `guide.dropz: Unknown mapping-guide key 'dropz'. Known keys: autoMatchByMapsTo, fieldMappings, taxonomyDirectives, drops.` A silently ignored typo would execute a plan you did not intend. - A `fieldMappings` entry requires a registered `transformName` - leave it off and the gate answers `Field transform '' is not registered.` The registered vocabulary is `SplitName`, `JoinName`, `CoerceStringToDateTime`, `CoerceDateTimeToString`, `CoerceIntegerToDouble`, and `CoerceDoubleToInteger`; for plain 1:1 renames against a hint, prefer inferred matching or a reviewed hand-written plan. - Transforms are type-checked against the target, not taken on faith: point `CoerceIntegerToDouble` at an integer target and the gate answers `Transform 'CoerceIntegerToDouble' produces output 0 of type Double, but target element 'ordersItemCount' is Integer.` ## Fan-out, gated on the ledger The same determinism makes downstream artifacts regenerable and diffable. One loop produces the JSON Schema and DDL editions of every contract in a directory: ```bash mkdir -p out for f in contracts/*.odcs.yaml; do name=$(basename "$f" .odcs.yaml) for target in jsonschema sql; do jq -n --rawfile s "$f" --arg t "$target" '{ sourceFormat: "odcs", sourceSchema: $s, targetFormat: $t, vendor: "postgres", targetHintFormat: "odcs", targetHintSchema: $s, mapping: { kind: "inferred" } }' \ | curl -sS -X POST \ "https://coremodels.example.com/graph/transform/schema/map/$PROJECT_ID" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ --data-binary @- > "out/$name.$target.result.json" done done ``` Then gate on lossiness kinds, not on `success` - `success: true` only means the transform ran. For contract fan-out, `SemanticNarrowing` entries for preserved `servers`/`quality` sections are expected steady state; a `StructuralDrop` or `TypeApproximation` is news: ```bash fail=0 for r in out/*.result.json; do bad=$(jq '[.lossiness[] | select(.kind == "StructuralDrop" or .kind == "TypeApproximation")] | length' "$r") if [ "$bad" -gt 0 ]; then echo "REVIEW: $r" jq -r '.lossiness[] | " [\(.kind)] \(.path): \(.explanation)"' "$r" fail=1 fi done exit $fail ``` A property someone types as `time` (approximated as DateTime, declared) or an out-of-enum `logicalType` trips this gate the day it lands in the contract, not the day a consumer notices. ## The steady state 1. **Author once** - derive or write the plan; read it; every heuristic is stamped with an `origin`, every intended omission is a `Drop` with a reason. 2. **Commit the artifacts** - plan and guide live next to the contract they govern. 3. **Replay on change** - CI calls `plan/execute`; the gate catches drift; determinism keeps diffs meaningful; the ledger keeps residual cost auditable. 4. **Regenerate deliberately** - when the contract truly evolves, re-derive and diff *the plan*: a smaller, more honest object than a diff of everything generated from it. The transform API documentation specifies the `schema/map` and `plan/execute` request shapes, the guide grammar, and the validation gate's contract.