# Google BigQuery — API: The CoreModels BigQuery Integration, Route by Route *The complete HTTP surface for governing a BigQuery estate: every verb, every role, every payload.* # The CoreModels BigQuery Integration, Route by Route *The complete HTTP surface for governing a BigQuery estate: every verb, every role, every payload.* The BigQuery connector in CoreModels (by ARAMAI) declares three capabilities - **Import**, **Audit**, and **Generate** - and the platform exposes them through two HTTP surfaces. The interactive surface under `graph/integrations/...` authenticates with your normal CoreModels login token and carries the working verb set: import, audit, reaudit, history, badge, generate, status, plus cross-vendor reconciliation and connector discovery (a sync-proposal surface also lives here, beyond this article's scope). The machine-to-machine surface under `v1/...` accepts user API keys and deliberately carries only what CI needs: audit and badge. This article walks the whole surface with real payloads. `$TOKEN` is your bearer credential; `$PROJECT_ID` is the 32-character hex id of the governing CoreModels project; the host is a placeholder. One posture rule governs everything below: import writes to the graph, additively, and nothing else writes at all. Audit and generate are read-only; recording an audit run in the history is opt-in bookkeeping, and only the reaudit verb records its run unconditionally. ## Discovery ```http GET https://coremodels.example.com/graph/integrations/vendors Authorization: Bearer $TOKEN ``` Any authenticated user can call this. It lists every registered connector with its key, display name, capability flags, and expected artifacts. The BigQuery entry looks like this: ```json { "key": "bigquery", "displayName": "Google BigQuery", "capabilities": "Import, Audit, Generate", "artifacts": { "information_schema": "required - JSON rows of INFORMATION_SCHEMA COLUMNS × TABLES (+ field-path descriptions, table options); documented query" } } ``` That `artifacts` map is the contract: one required artifact, `information_schema`, produced by the documented per-dataset extraction query (see the quickstart for the SQL). A fourth capability, `LiveSync`, exists in the model but is deliberately deferred - CoreModels never holds your Google credentials, so every verb below works on artifacts you extract yourself. ## Import - `POST graph/integrations/bigquery/import/{projectId}` (Admin) ```http POST https://coremodels.example.com/graph/integrations/bigquery/import/$PROJECT_ID Authorization: Bearer $TOKEN Content-Type: application/json { "artifacts": { "information_schema": "" } } ``` The body is an `ArtifactsRequest`: `artifacts` maps artifact name to raw content; an optional `spaces` array targets specific space ids (empty means the project's main space). Import is additive - already-governed nodes are never mutated; drift is the audit's job. The response returns `success`, `vendor`, `projectName`, then counters (`datasetsAdded`, `datasetsSkippedExisting`, `fieldsAdded`, `lineageEdgesAdded`, `lineageEdgesSkipped`, `nodesEnriched`), a `snapshotStored` flag, and the two channels every verb shares: `lossiness` (what was approximated, on success) and `errors` (what prevented progress). For BigQuery, lineage counters are always zero - the `information_schema` extract carries no lineage. Omitting the artifacts is a structured failure, not a 500: ```json { "success": false, "lossiness": [], "errors": [ { "path": "artifacts", "message": "Body must include 'artifacts': { \"\": \"\" } (e.g. manifest for dbt)." } ] } ``` An unknown vendor key fails the same way, with the message listing every registered key. ## Audit - `POST graph/integrations/bigquery/audit/{projectId}` (Viewer) Same body shape as import, plus one flag: `recordHistory` (default `false`) appends the run to the rolling audit trail with trigger `audit`. Without it, the audit verb stays strictly read-only. The response is the full audit report: ```json { "success": true, "vendor": "bigquery", "projectName": "my-gcp-project", "errorCount": 0, "warningCount": 1, "infoCount": 3, "codes": { "semi-structured-column": 1, "table-no-description": 3 }, "driftedObjects": [], "fingerprint": "9c2f4e1a0b7d3c58", "metrics": { "Datasets (estate)": "12", "Datasets governed": "12 / 12", "Fields governed": "84 / 84", "Governed nodes with canonical mappings": "0 / 96 (0%)", "Last import": "2026-07-20T09:14:22.6182740+00:00" }, "findings": [ { "section": "Conformance", "severity": "Info", "code": "table-no-description", "subject": "my-gcp-project.analytics.raw_events", "message": "Table/view has no description - undocumented datasets resist governance and agent grounding.", "detail": null } ], "markdown": "", "historyRecorded": false, "lossiness": [] } ``` Findings are grouped in three sections - `Coverage` (codes `dataset-unmapped`, `field-unmapped`), `Drift` (`dataset-removed`, `field-removed`, `field-type-drift`, `enum-constraint-removed`, `enum-narrowed`, `enum-widened`, `contract-drift`), and `Conformance`, where BigQuery contributes `table-no-description` (Info) and `semi-structured-column` (Warning). `errorCount > 0` is the CI-gate fail condition. `metrics` is a string map of the headline numbers - estate size, governed coverage, how many governed nodes carry a non-vendor canonical mapping, and when the estate was last imported. The `fingerprint` is a short content hash of the artifact, so two runs with the same fingerprint audited byte-identical extracts. ## Reaudit - `POST graph/integrations/bigquery/reaudit/{projectId}` (Viewer) The audit above asks "do these fresh artifacts still conform to the governed model?" Reaudit asks the mirror question: *the governed model changed - does the last-known estate still conform?* It runs the same audit engine over the estate snapshot stored at import time against the **current** governed model. No artifacts needed: ```http POST https://coremodels.example.com/graph/integrations/bigquery/reaudit/$PROJECT_ID Authorization: Bearer $TOKEN Content-Type: application/json {} ``` The optional body fields are `projectName` (which stored snapshot to use; `null` means the latest), `spaces`, and `notifySlack` (post to the project's configured Slack webhook when the run has errors or warnings; default `false`). Unlike audit, reaudit **always** records its run in the history, with trigger `reaudit`. It fails honestly when no snapshot is available - for example when the import reported `snapshotStored: false` because the encoded snapshot exceeded the storage cap (~1.5 MB). ## History - `GET graph/integrations/bigquery/history/{projectId}` (Viewer) The rolling audit trail, newest first, one entry per recorded run: ```json { "success": true, "vendor": "bigquery", "projects": [ { "projectName": "my-gcp-project", "runs": [ { "at": "2026-07-29T06:41:00.4413094+00:00", "trigger": "reaudit", "errorCount": 1, "warningCount": 1, "infoCount": 3, "codes": { "field-type-drift": 1, "semi-structured-column": 1, "table-no-description": 3 }, "fingerprint": "9c2f4e1a0b7d3c58" } ] } ] } ``` The `trigger` values tell you how each run happened: `audit` (interactive, `recordHistory: true`), `ci` (the v1 surface), `reaudit`, or `scheduled` (the optional server-side re-audit heartbeat). Timestamps are UTC ISO-8601 and sort lexicographically. ## Badge - `GET graph/integrations/bigquery/badge/{projectId}` (Viewer) Returns `image/svg+xml` - a shields-style badge labeled `bigquery audit`, rendered from the latest recorded run: green clean, yellow warnings only, red errors, gray when nothing is recorded. Even an unknown vendor key returns a gray badge rather than a JSON error, so whatever consumes the badge always gets an image. ## Generate - `POST graph/integrations/bigquery/generate/{projectId}` (Viewer) BigQuery is a full-loop connector: the governed model generates DDL back out. ```http POST https://coremodels.example.com/graph/integrations/bigquery/generate/$PROJECT_ID Authorization: Bearer $TOKEN Content-Type: application/json { "typeNames": [] } ``` `typeNames` restricts generation to named types (empty means everything eligible); `targetVersion` and `extra` exist in the request shape for vendors that need dialect switches - BigQuery currently needs none. The response: ```json { "success": true, "artifacts": [ { "name": "coremodels_bigquery_tables.sql", "kind": "sql", "content": "-- Generated by CoreModels - governed BigQuery table definitions.\n..." } ], "lossiness": [ { "kind": "StructuralDrop", "path": "daily_summary", "explanation": "Views are derived objects; DDL generation covers tables only." } ], "errors": [] } ``` One artifact comes back: `CREATE TABLE IF NOT EXISTS` statements with `NOT NULL` from governed checks and `OPTIONS(description=...)` on both columns and tables. BigQuery enforces no primary keys, foreign keys, or CHECK constraints, so governed taxonomies and references are written into the descriptions - where BigQuery users actually read - never faked as constraints. Views are skipped with a declared lossiness record, as above. If nothing is eligible, generate fails with an explicit error rather than emitting an empty file. ## Status - `GET graph/integrations/bigquery/status/{projectId}` (Viewer) ```json { "success": true, "vendor": "bigquery", "imported": true, "state": { "...": "last-import bookkeeping: versions, timestamps, fingerprint, counts" }, "governedDatasets": 10 } ``` `imported` tells you whether an import state exists at all; `governedDatasets` counts vendor-identified datasets currently resolvable as governed Types. ## Reconcile - `POST graph/integrations/reconcile/{projectId}` (Admin) The one route that is not vendor-scoped. If two connectors govern the same physical relation - say a BigQuery table and the transformation-tool model that materializes it - reconciliation links the pairs as one entity via reciprocal sameAs assertions. Body: `{ "vendorA": "bigquery", "vendorB": "" }`. It is idempotent and reports the matched pairs, the fields they matched on, and what remained unmatched on each side. ## The v1 machine surface Two routes, both Viewer role, both working with user API keys - this is what CI and scripts should call: - `POST v1/{projectId}/integrations/bigquery/audit` - same body as the interactive audit; the response is wrapped in the `ApiResponse` envelope, so every field above lives under `data.*` (`data.errorCount > 0` means fail the build). With `recordHistory: true` the run lands in the history with trigger `ci`. - `GET v1/{projectId}/integrations/bigquery/badge` - the same SVG badge for READMEs and dashboards. Reaudit and history intentionally stay on the interactive surface; the v1 integration surface carries exactly the two verbs automation needs - the gate and its badge. That is the working surface - the interactive verbs above, two machine routes, one connector key - everything the import–audit–generate loop needs. For the extraction recipe and a step-by-step first run, see the Google BigQuery quickstart in the CoreModels documentation.