# cBioPortal — Automation: One character, one broken study: automating cBioPortal drift checks in CI Here is a pull request diff from a study repository: # One character, one broken study: automating cBioPortal drift checks in CI Here is a pull request diff from a study repository: ```diff -#STRING NUMBER STRING STRING +#STRING STRING STRING STRING ``` One token changed on one header row. This is version two of a published study - the routine update every active cohort ships - and nothing in the pipeline will object. A standard review scans past it. validateData.py, rightly the format authority, checks that v2 is a well-formed study, not that v2 still means what v1 meant. Incremental loading moves the files without asking either. No tool diffs a study update against the published version's semantics - so the attribute that used to be numeric is now text, and every consumer that treated it as a number - a cohort filter, a survival plot, an export into an analysis pipeline - is wrong in a way that surfaces weeks later as a bug report about a chart. A drift gate answers that diff in ninety seconds, before the importer runs. Retype a governed attribute and the audit returns `field-type-drift`, naming both types; drop or rename one and it returns `field-removed` - the semantic diff of v2 against governed meaning, delivered on the pull request. The same gate catches what the portal's validator would only reject later: an attribute ID that is not UPPER_CASE comes back as `attribute-id-not-upper`, and the workflow's fail-on setting decides whether warnings like it merely annotate the run or fail it outright. This article builds that gate for a cBioPortal study repository, then adds the three pieces that turn a one-shot check into a system: a badge, a rolling history, and re-audit - including on a schedule. ## The route CI calls Automation uses the machine-to-machine surface, which accepts revocable user API keys instead of interactive login tokens: ``` POST https://coremodels.example.com/v1/{projectId}/integrations/cbioportal/audit ``` It is Viewer role, because the audit never writes model shape. The response is wrapped in the standard API envelope, so every field of the report sits under `data`: ```json { "success": true, "data": { "vendor": "cbioportal", "projectName": "example_brca_2026", "errorCount": 1, "warningCount": 1, "infoCount": 1, "codes": { "field-type-drift": 1, "attribute-id-not-upper": 1, "attribute-no-description": 1 }, "driftedObjects": ["patient.AGE"], "fingerprint": "c4a71f2e9b380d56", "findings": [ { "section": "Drift", "severity": "Error", "code": "field-type-drift", "subject": "patient.AGE", "message": "Field type changed since the last import.", "detail": "governed: NUMBER, estate: STRING" }, { "section": "Conformance", "severity": "Info", "code": "attribute-no-description", "subject": "patient", "message": "1 clinical attribute(s) carry no description row - curators downstream will guess.", "detail": "OS_STATUS" }, { "section": "Conformance", "severity": "Warning", "code": "attribute-id-not-upper", "subject": "sample.cancerType", "message": "Attribute IDs must be UPPER_CASE for cBioPortal validation to pass.", "detail": null } ], "markdown": "…", "historyRecorded": true, "lossiness": [] } } ``` The gate is one comparison: `data.errorCount > 0` means this pull request violates governed meaning. Fail the build. ## The workflow Three settings do all the configuration: a repository secret `COREMODELS_API_KEY` holding a user API key with Viewer access to the governing project, and two repository variables, `COREMODELS_API_URL` and `COREMODELS_PROJECT_ID`. ```yaml name: CoreModels schema audit on: pull_request: paths: - "**/data_clinical_*.txt" - "**/meta_study.txt" permissions: contents: read pull-requests: write jobs: audit: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Audit the study schema against CoreModels env: COREMODELS_API_URL: ${{ vars.COREMODELS_API_URL }} COREMODELS_API_KEY: ${{ secrets.COREMODELS_API_KEY }} COREMODELS_PROJECT_ID: ${{ vars.COREMODELS_PROJECT_ID }} FAIL_ON: error # set to 'warning' to gate harder run: | set -euo pipefail # Header block only - data rows are ignored by the connector anyway. head -5 data_clinical_patient.txt > patient.head head -5 data_clinical_sample.txt > sample.head jq -n \ --rawfile patient patient.head \ --rawfile sample sample.head \ --rawfile meta meta_study.txt \ '{artifacts: {clinical_patient: $patient, clinical_sample: $sample, meta: $meta}, recordHistory: true}' > audit-request.json status=$(curl -sS -o audit-response.json -w "%{http_code}" -X POST \ "$COREMODELS_API_URL/v1/$COREMODELS_PROJECT_ID/integrations/cbioportal/audit" \ -H "Authorization: Bearer $COREMODELS_API_KEY" \ -H "Content-Type: application/json" \ --data-binary @audit-request.json) if [ "$status" != "200" ]; then echo "::error::CoreModels audit call failed with HTTP $status" head -c 2000 audit-response.json || true exit 1 fi if [ "$(jq -r '.success' audit-response.json)" != "true" ]; then echo "::error::CoreModels audit returned an error: $(jq -r '.error.message // "unknown"' audit-response.json)" exit 1 fi errors=$(jq -r '.data.errorCount' audit-response.json) warnings=$(jq -r '.data.warningCount' audit-response.json) drifted=$(jq -r '(.data.driftedObjects // []) | join(",")' audit-response.json) jq -r '.data.markdown' audit-response.json >> "$GITHUB_STEP_SUMMARY" # One annotation per finding, on the right severity channel. jq -r '.data.findings[] | "\(.severity)|\(.code)|\(.subject)|\(.message)"' audit-response.json | while IFS='|' read -r severity code subject message; do case "$severity" in Error) echo "::error title=$code::$subject - $message" ;; Warning) echo "::warning title=$code::$subject - $message" ;; esac done if [ -n "$drifted" ]; then echo "Drifted objects: $drifted"; fi fail=0 if [ "$errors" -gt 0 ]; then fail=1; fi if [ "$FAIL_ON" = "warning" ] && [ "$warnings" -gt 0 ]; then fail=1; fi if [ "$fail" = "1" ]; then echo "::error::This change violates governed meaning in CoreModels. See the audit report." exit 1 fi ``` Points worth calling out. The `paths` filter keeps the job off pull requests that do not touch the schema. `head -5` sends only the header block, so patient data never leaves the runner even though the audit is schema-only anyway. `jq --rawfile` handles the tab and newline escaping and keeps the file contents out of `argv`. The markdown report goes to the job summary, and the per-finding annotations attach the exact code and subject to the run. The CoreModels repository also ships a composite `CoreModels Schema Audit` action, but its inputs (`manifest-path`, `catalog-path`) build a dbt-shaped request body. For a cBioPortal study the direct step above is the right shape - it posts to the same `v1` route with the clinical artifact names. ## What `errorCount` actually counts for this connector Findings sort into three sections - Coverage, Drift, Conformance - at three severities, and only Error increments `errorCount`. For cBioPortal in practice: | Code | Section | Severity | Fires when | |---|---|---|---| | `field-type-drift` | Drift | Error | a datatype row changed for a governed attribute | | `field-removed` | Drift | Error | a governed attribute is gone from a submitted instrument's file | | `dataset-unmapped` | Coverage | Warning | an instrument in the files is not governed yet | | `field-unmapped` | Coverage | Info | an attribute was added since the last import | | `attribute-id-not-upper` | Conformance | Warning | an attribute ID is not `UPPER_CASE` | | `attribute-no-description` | Conformance | Info | attributes on an instrument have no description row | So a default gate fails only on genuine meaning changes: retyped, removed, or vanished. The two cBioPortal conformance rules are advisory by design - `attribute-id-not-upper` is the one that will break portal validation later, which is why teams commonly flip `FAIL_ON` to `warning` once the existing attribute IDs are clean. Three absences are worth knowing. `contract-drift` cannot fire here, because staging files carry no contract-enforcement flag. The `enum-*` codes only apply where a governed attribute has been given a controlled value list by hand - clinical staging files declare no accepted values, so a hand-added taxonomy will report `enum-constraint-removed` (Warning) on every audit until someone reconciles the two. And the shared `dataset-removed` code, which fires for warehouse estates when a whole governed table vanishes, does not apply to this connector: removal is scoped by identity namespace, and a cBioPortal instrument identity (`patient`, `sample`) *is* its own namespace, so an instrument you did not submit is simply out of scope for the comparison. That scoping is a feature for CI. Auditing a partial artifact set is safe: send only `clinical_patient` on a pull request that only touches the patient file, and the governed `sample` instrument is left alone rather than reported as missing. ## The badge Every recorded run feeds an SVG status badge, available on both surfaces: ```bash curl -sS "https://coremodels.example.com/v1/$PROJECT_ID/integrations/cbioportal/badge" \ -H "Authorization: Bearer $COREMODELS_API_KEY" -o schema-audit.svg ``` It is labeled `cbioportal audit` and color-coded from the latest recorded run: green when clean, yellow when the run had warnings only, red with the error count when it had errors, gray when nothing has been recorded yet. The route is authenticated like every other route, so a bare image tag in a public README will get a `401` - fetch the SVG in the workflow and publish it where your readers are, or render it behind whatever proxy already fronts your internal docs. The badge is only as current as your history, which is why `recordHistory: true` in the gate matters: it is the call that gives the badge something to show. ## The rolling history ```bash curl -sS "https://coremodels.example.com/graph/integrations/cbioportal/history/$PROJECT_ID" \ -H "Authorization: Bearer $TOKEN" | jq '.projects[].runs' ``` ```json [ { "at": "2026-08-03T11:02:47.1180930+00:00", "trigger": "ci", "errorCount": 1, "warningCount": 1, "infoCount": 1, "codes": { "field-type-drift": 1, "attribute-id-not-upper": 1, "attribute-no-description": 1 }, "fingerprint": "c4a71f2e9b380d56" }, { "at": "2026-08-02T22:15:03.4470010+00:00", "trigger": "scheduled", "errorCount": 0, "warningCount": 1, "infoCount": 1, "codes": { "attribute-id-not-upper": 1, "attribute-no-description": 1 }, "fingerprint": "5b19ee74c0a3f2d8" } ] ``` Runs are newest first, one trail per vendor-side study, capped at the 50 most recent. Each record is deliberately compact - timestamp, trigger, counts, codes, fingerprint - never the full findings. The trail exists to answer "is this study drifting over time?", not "what exactly is wrong right now", which is what a live audit is for. Triggers are `audit` (interactive), `ci` (the API-key route), `reaudit`, and `scheduled`. The `fingerprint` is a content hash of the audited artifacts, so identical fingerprints across runs mean the files did not change - useful when a run's counts move but the files did not, which is exactly the signal that the *model* changed. ## Re-audit: drift from the other side The gate answers whether fresh files still conform to the governed model. The opposite question - does the governed model still match the last-known study? - matters when the model changes rather than the files. Re-audit runs the same engine over the snapshot stored at import time against the current governed view. No artifacts, no credentials: ```bash curl -sS -X POST \ "https://coremodels.example.com/graph/integrations/cbioportal/reaudit/$PROJECT_ID" \ -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \ -d '{}' | jq '{errorCount, warningCount, codes}' ``` It always records its run, so it moves the badge and shows in the history like any other. It lives on the interactive surface only. Its one prerequisite is a stored snapshot: if the import reported `snapshotStored: false` - which happens when a very large study pushes the encoded snapshot past the storage cap - fresh-artifact audits still work perfectly, but there is nothing for re-audit to run against, and the call says so. A natural place to call it is the workflow that merges governance changes, or a nightly job in the repository that owns the CoreModels project. ## The scheduled heartbeat Re-audit can also run server-side without any repository at all. CoreModels includes a config-gated recurring worker that re-audits opted-in projects against their stored snapshots and appends each run to the same history, with the trigger `scheduled`. It is off by default and enabled per project by a CoreModels operator, because it writes history entries. Turned on, it changes the failure mode of the whole system. Without it, model-side drift stays invisible until the next pull request touches the study - which for a completed cohort might be never. With it, the badge goes yellow or red on its own cadence, and the history shows the date the governed model and the last-known study parted ways. The CI gate catches file-side drift, the heartbeat catches model-side drift, the history records both, and the badge is the one-glance summary of where things stand. For the extraction recipe behind the request bodies above, see the cBioPortal quickstart in the CoreModels integration docs.