# AWS Glue — API: The AWS Glue Integration API, Route by Route CoreModels models vendor governance as a small, fixed set of verbs and applies them uniformly to every connector. For AWS Glue Data Catalog - vendor key `glue` - that is eight core routes on the interactive surface and two on the machine-to-machine surface. (A newer sync-plan surface - `sync/propose`, plan fetch, and the sync ledger - sits beside these and is not covered here.) This is the reference: the exact paths, the role each one enforces, the request bodies as the API defines them, and the responses you actually get back. # The AWS Glue Integration API, Route by Route CoreModels models vendor governance as a small, fixed set of verbs and applies them uniformly to every connector. For AWS Glue Data Catalog - vendor key `glue` - that is eight core routes on the interactive surface and two on the machine-to-machine surface. (A newer sync-plan surface - `sync/propose`, plan fetch, and the sync ledger - sits beside these and is not covered here.) This is the reference: the exact paths, the role each one enforces, the request bodies as the API defines them, and the responses you actually get back. Both surfaces authenticate with `Authorization: Bearer …`. The interactive surface (`graph/integrations/…`) expects your normal CoreModels login token; the `v1` surface accepts user API keys, which is what makes it the right target for pipelines and dashboards. Throughout, `https://coremodels.example.com` stands in for your API base URL, `$TOKEN` for the token, and `$PROJECT_ID` for the 32-character hex project id. | Route | Role | Writes? | |---|---|---| | `GET graph/integrations/vendors` | any authenticated | no | | `POST graph/integrations/glue/import/{projectId}` | Admin | yes - additive only | | `POST graph/integrations/glue/audit/{projectId}` | Viewer | history, only if asked | | `POST graph/integrations/glue/reaudit/{projectId}` | Viewer | history, always | | `GET graph/integrations/glue/history/{projectId}` | Viewer | no | | `GET graph/integrations/glue/badge/{projectId}` | Viewer | no | | `POST graph/integrations/glue/generate/{projectId}` | Viewer | no | | `GET graph/integrations/glue/status/{projectId}` | Viewer | no | | `POST v1/{projectId}/integrations/glue/audit` | Viewer (API key) | history, only if asked | | `GET v1/{projectId}/integrations/glue/badge` | Viewer (API key) | no | ## Three request bodies, reused everywhere The artifact-bearing routes (import and audit, on both surfaces) share one shape: ```json { "artifacts": { "tables": "" }, "spaces": [], "recordHistory": false } ``` `artifacts` maps artifact name to raw content; Glue defines a single name, `tables`. `spaces` is optional - an empty array means the project's main space. `recordHistory` applies to audit only. Re-audit takes a three-field body, everything optional: ```json { "projectName": null, "spaces": null, "notifySlack": false } ``` `notifySlack` opts in to a notification on the project's configured Slack webhook when the recorded run carries errors or warnings - like recording, an explicit ask, default `false`. Generate takes: ```json { "typeNames": [], "targetVersion": null, "extra": {}, "spaces": null } ``` `typeNames` restricts generation to named types (empty means everything eligible). `targetVersion` and `extra` exist for connectors with dialect switches; the Glue generator reads neither, so for this vendor they are inert. ## Discovery ```bash curl -sS -H "Authorization: Bearer $TOKEN" \ "https://coremodels.example.com/graph/integrations/vendors" ``` Any authenticated user may call it. The Glue entry is your capability contract: ```json { "key": "glue", "displayName": "AWS Glue Data Catalog", "capabilities": "Import, Audit, Generate", "artifacts": { "tables": "required - `aws glue get-tables` JSON (one response, an array of responses, or a bare TableList)" } } ``` Glue declares all three of Import, Audit and Generate. (A fourth flag, live sync, exists in the capability vocabulary and is deliberately not claimed by any connector - CoreModels never holds your cloud credentials.) ## Import - Admin ```bash curl -sS -X POST \ "https://coremodels.example.com/graph/integrations/glue/import/$PROJECT_ID" \ -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ --data-binary @import-request.json ``` Import is additive by design: tables become governed Types, columns and partition keys become Elements, vendor metadata is attached - and an already-governed node is never mutated or deleted. Re-import a grown catalog and the new tables land while the familiar ones are counted under `datasetsSkippedExisting`; new columns on already-governed tables land under `fieldsAdded`. Drift is not applied here. Drift is the audit's job, and acting on it is a human decision. The response carries `datasetsAdded`, `datasetsSkippedExisting`, `fieldsAdded`, `lineageEdgesAdded`, `lineageEdgesSkipped`, `nodesEnriched`, a `snapshotStored` flag, and the `lossiness` / `errors` arrays. Errors mean *could not proceed*; lossiness means *proceeded, and here is what was approximated*. ## Audit - Viewer ```bash curl -sS -X POST \ "https://coremodels.example.com/graph/integrations/glue/audit/$PROJECT_ID" \ -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ --data-binary @audit-request.json ``` Read-only. `recordHistory: true` is the one opt-in write it will make, appending a compact record of the run to the project's rolling trail as bookkeeping - never as governed meaning. The response reports `errorCount` (greater than zero means governed meaning is violated), `warningCount`, `infoCount`, a `codes` map of finding code to count, `driftedObjects`, `fingerprint` (a content hash of the artifact), `metrics`, the itemized `findings`, a `markdown` rendering, `historyRecorded`, and `lossiness`. ## Re-audit - Viewer ```bash curl -sS -X POST \ "https://coremodels.example.com/graph/integrations/glue/reaudit/$PROJECT_ID" \ -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ -d '{}' ``` Same engine, different input. The live audit asks *"do these fresh artifacts still conform to the governed model?"*; re-audit asks *"does the governed model still match the last-known estate?"* by replaying the audit over the catalog snapshot stored at import time. No artifacts, no AWS access. Pass `projectName` to target a specific vendor-side database when several have been imported; `null` uses the most recently stored snapshot. Unlike audit, re-audit **always** records its run, and the report carries an extra `Snapshot stored` metric telling you how old that snapshot is. If no snapshot exists, the call fails with a precise error: `No stored estate snapshot for vendor 'glue' - import the vendor project first (imports persist the parsed snapshot).` ## History - Viewer ```bash curl -sS -H "Authorization: Bearer $TOKEN" \ "https://coremodels.example.com/graph/integrations/glue/history/$PROJECT_ID" ``` ```json { "success": true, "vendor": "glue", "projects": [ { "projectName": "lake", "runs": [ { "at": "2026-08-04T09:41:07.6620114+00:00", "trigger": "ci", "errorCount": 0, "warningCount": 1, "infoCount": 2, "codes": { "semi-structured-column": 1, "table-no-description": 1, "classification-missing": 1 }, "fingerprint": "3f9c1d5a7b2e4086" } ] } ] } ``` Runs are newest first, grouped per vendor-side database, and the window is capped - the trail is a trend line, not an archive. `trigger` is one of `audit`, `ci`, `reaudit`, or `scheduled`. ## Badge - Viewer ```bash curl -sS -H "Authorization: Bearer $TOKEN" \ "https://coremodels.example.com/graph/integrations/glue/badge/$PROJECT_ID" > glue-audit.svg ``` Returns `image/svg+xml`: a shields-style badge labeled `glue audit`, rendered from the latest **recorded** run. Green is clean, yellow is warnings only, red is errors, gray means no recorded runs yet (you will also get gray for an unrecognized vendor key rather than an error page). A project whose audits never set `recordHistory` keeps a gray badge forever. ## Generate - Viewer Glue claims the Generate capability, and the artifact is worth having: ```bash curl -sS -X POST \ "https://coremodels.example.com/graph/integrations/glue/generate/$PROJECT_ID" \ -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ -d '{ "typeNames": [] }' ``` ```json { "success": true, "artifacts": [ { "name": "coremodels_glue_tables.sql", "kind": "sql", "content": "-- Generated by CoreModels …" } ], "lossiness": [], "errors": [] } ``` The `content` is Athena/Hive DDL built from the governed model: ```sql -- Generated by CoreModels - governed Athena/Glue table definitions. -- Set the LOCATION per table before running; meaning changes belong in CoreModels. CREATE EXTERNAL TABLE IF NOT EXISTS events ( event_id string COMMENT 'Event key.', user_id bigint, occurred_at timestamp, payload struct COMMENT 'Raw payload.' ) COMMENT 'Raw product events landed from Kinesis.' PARTITIONED BY ( event_date date COMMENT 'Partition.' ) STORED AS PARQUET LOCATION 's3:///events/'; ``` Note what rides along: the recorded Hive type is re-emitted verbatim (a `struct<…>` column comes back as a `struct<…>` column, not as a degraded `string`), governed descriptions and taxonomy allowed-values become column `COMMENT`s, partition-flagged elements are reconstructed into `PARTITIONED BY`, and the `LOCATION` is a placeholder you must set - the governed model knows your schema, not your bucket layout. Two skips are declared rather than silent: governed views (`Views are derived objects; DDL generation covers tables only.`) and governed types with no elements, both reported as `StructuralDrop` records in `lossiness`. If nothing is eligible at all, the call returns `success: false` with `No eligible tables found to generate DDL for.` ## Status - Viewer ```bash curl -sS -H "Authorization: Bearer $TOKEN" \ "https://coremodels.example.com/graph/integrations/glue/status/$PROJECT_ID" ``` ```json { "success": true, "vendor": "glue", "imported": true, "state": { "vendor": "glue", "projectName": "lake", "importedAt": "2026-08-04T09:14:22.1043117+00:00", "toolVersion": null, "artifactVersion": null, "generatedAt": null, "sourceFingerprint": "3f9c1d5a7b2e4086", "counts": "fieldsAdded=0, lineageAdded=0, lineageSkipped=0, nodesEnriched=10", "facts": "{\"tables\":\"2\",\"views\":\"0\"}" }, "governedDatasets": 2 } ``` `imported: false` with a null `state` simply means no Glue import has happened in this project. The tool-version, artifact-version and generated-at slots of the state record stay empty for Glue: a `get-tables` response carries no generator stamp, and we do not fabricate one. `facts` is the connector's own parser summary - how many tables and how many views it saw. ## The machine-to-machine surface ```bash curl -sS -X POST \ "https://coremodels.example.com/v1/$PROJECT_ID/integrations/glue/audit" \ -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \ --data-binary @audit-request.json ``` Identical request body, one structural difference in the response: it is wrapped in the standard API envelope, so every report field lives under `data.*`. A pipeline checks `data.errorCount`, not `errorCount`. When `recordHistory` is true here the run is recorded with trigger `ci`, which keeps automated runs distinguishable from interactive ones in the trail. The badge is available on this surface too, at `GET v1/{projectId}/integrations/glue/badge`. What is *not* here is as important: `reaudit` and `history` live on the interactive surface only. ## Failure modes worth handling An unknown vendor key on any route returns `success: false` with a message that names the alternatives: `Unknown vendor 'gluu'. Registered: …`. A missing or empty artifact bag returns `Body must include 'artifacts': { "": "" } (e.g. manifest for dbt).` A payload that parses as JSON but is not shaped like `get-tables` output fails at the connector with `The artifact is valid JSON but not shaped like get-tables output: …`, and one with no named tables with `No tables found (expected TableList entries).` The posture behind the whole table at the top: import is the only verb that writes governed structure, and even it only adds; audit and generate write nothing at all. That is why a Viewer-scoped key is all your automation ever needs. (One further Admin verb sits on the shared surface - `POST graph/integrations/reconcile/{projectId}` - which links datasets that two vendor estates govern as the same physical relation; it becomes interesting once Glue is not your only connected estate.) For a task-shaped walk through the same routes, see the AWS Glue quickstart in the CoreModels documentation.