# cBioPortal — Deep dive: Following one clinical attribute all the way into the graph

Take a single column out of a cBioPortal patient staging file:

# Following one clinical attribute all the way into the graph

Take a single column out of a cBioPortal patient staging file:

```text
#Age
#Age at diagnosis in years
#NUMBER
#1
AGE
```

Five cells. By the time CoreModels has finished importing the study, those five cells have become a
governed Element with a floating-point value type, a vendor identity that survives renaming, a
metadata record holding the literal string `NUMBER` and the display name `Age`, an entry in a
persisted estate snapshot, and a line item in every future drift audit. This article traces that
path for every construct the connector recognizes - what the parser extracts, where each fact lands,
which audit codes can fire against it, and exactly where meaning is approximated or dropped.

## The parse contract

The connector reads a clinical file as: every leading line that starts with `#` is a header row;
the first non-blank line after them is the attribute-ID row; everything below is data and is
discarded. Fewer than four header rows, or no attribute row, and the file is rejected outright with
`Not a clinical staging file (needs 4 '#' header rows above the attribute-ID row).`

Column by column, the mapping is positional:

| Header row | Becomes |
|---|---|
| 0 - display names | `cbio.displayName` in the field's meta bag |
| 1 - descriptions | the field's description |
| 2 - datatypes | the field's native type, upper-cased; empty cell defaults to `STRING` |
| 3 - priorities | `cbio.priority` in the field's meta bag |
| attribute-ID row | the field's name - the identity everything else keys off |

Rows are read defensively. A header row shorter than the attribute row yields empty cells for the
columns it does not reach - which is why an attribute with no description is a normal parse outcome
that raises a conformance finding rather than an error. Blank attribute IDs are skipped; duplicates
are skipped case-insensitively, first occurrence winning. A file yielding no usable attributes is
treated as not-a-staging-file. One trap: *every* leading `#` line counts as a header row, so a
comment line pasted above the block shifts the positional mapping.

Each file becomes one dataset in the neutral estate model: vendor id `patient` or `sample`, kind
`Table`, materialization `clinical`, description `cBioPortal clinical {instrument} attributes`. The
payload is fingerprinted with a truncated SHA-256, so two submissions compare cheaply.

Two structural facts are recognized during the parse. `PATIENT_ID` in the patient file and
`SAMPLE_ID` in the sample file each gain two normalized checks, `Unique` and `NotNull`, both named
`identity`. And the sample file's `PATIENT_ID` is rewritten into a reference: it keeps its
description and datatype, gains a `Relationship` check, and points at the patient instrument's
`PATIENT_ID`. That rewrite only happens when a patient dataset is present in the same submission -
audit a sample file on its own and the column stays a plain string.

The optional meta file contributes three keys and nothing else: `cancer_study_identifier` becomes
the study name (defaulting to `cbioportal-study` when absent), `name` becomes a `studyName` fact,
`type_of_cancer` becomes a `cancerType` fact. Two more facts are derived: `instruments` and
`attributes`.

## Identity: what the graph keys on

Node ids in CoreModels must be camelCase alphanumerics, so vendor identities are sanitised on the
way in and preserved elsewhere. For our study:

| Vendor identity | Graph node id | Node kind |
|---|---|---|
| `patient` | `patient` | Type |
| `sample` | `sample` | Type |
| `patient.AGE` | `patientAGE` | Element |
| `patient.PATIENT_ID` | `patientPATIENTID` | Element |
| `sample.cancerType` | `sampleCancerType` | Element |

The id is a convenience. The identity that matters is the `mapsTo` assertion written on every node -
a two-property mixin value whose standard is the vendor key and whose uri is the vendor identity:

```json
{ "transformMapsToStandard": "cbioportal", "transformMapsToUri": "patient.AGE" }
```

That is the same carrier every other CoreModels import writes, so a vendor identity survives a
format export intact. Audits match on it, never on labels - so an SME can relabel `AGE` to
`Age at diagnosis` and the next audit still recognizes the same governed attribute. It also means a
genuine rename in the staging file is real drift, which is correct: renaming an attribute ID breaks
every consumer keyed on it.

One consequence deserves a warning label. The dataset identity is the bare instrument name,
`patient` or `sample`, not something study-scoped - so two studies imported into one project claim
the same identities. Govern one study per project. Sanitisation is also collision-checked: two
distinct vendor identities that would collapse onto one node id make the import refuse and name
both, rather than silently merging two attributes.

## Two write paths

Schema shape - Types, Elements, value types, references - flows through a snapshot-to-IR mapper and
out through the shared CoreModels writer, the same path every other format import takes. Estate
facts the IR deliberately cannot hold are written natively afterwards.

| Estate fact | Where it lands |
|---|---|
| clinical file | Type node, label = instrument name |
| clinical attribute | Element on that Type |
| `NotNull` from an identity check | a `Required` element fact, stashed on the shared element-facts mixin |
| sample's `PATIENT_ID` | expected-type relation to the patient Type |
| vendor identity | `mapsTo` mixin value `{cbioportal, <vendor id>}` |
| native type, description, display name, priority, checks, materialization | the `cBioPortal Metadata` mixin |
| study id/name/cancer type, counts, fingerprint, timestamp | the integration state node |
| parsed estate | the estate snapshot node (gzip+base64, size-capped) |
| recorded audit runs | the audit history node (rolling, 50 runs) |

The metadata mixin is where the vendor's own vocabulary survives. Its property ids are the vendor
key plus the property name, so the value written for `patient.AGE` reads:

```json
{
  "cbioportalDataType": "NUMBER",
  "cbioportalDescription": "Age at diagnosis in years",
  "cbioportalTags": "",
  "cbioportalMeta": "{\"cbio.displayName\":\"Age\",\"cbio.priority\":\"1\"}",
  "cbioportalChecks": ""
}
```

And for `patient.PATIENT_ID`, whose identity checks are normalized:

```json
{
  "cbioportalDataType": "STRING",
  "cbioportalDescription": "Unique patient identifier",
  "cbioportalTags": "",
  "cbioportalMeta": "{\"cbio.displayName\":\"Patient Identifier\",\"cbio.priority\":\"1\"}",
  "cbioportalChecks": "[{\"kind\":\"Unique\",\"name\":\"identity\",\"severity\":null,\"detail\":null},{\"kind\":\"NotNull\",\"name\":\"identity\",\"severity\":null,\"detail\":null}]"
}
```

The instrument Type carries the dataset-level twin: `cbioportalUniqueId` (`patient`),
`cbioportalResourceKind` (`table`), `cbioportalMaterialization` (`clinical`),
`cbioportalContract` (`false`), and empty strings for the properties cBioPortal has no concept of -
`cbioportalRelationName`, `cbioportalAccess`. Every property is rewritten on every import, with `""`
for absent, so a fact that disappears from the estate cannot linger as stale metadata.

That refresh is the one thing an import updates in place, and it is deliberate: mixin values mirror
the estate, they are not governed meaning. Governed nodes themselves are never mutated or deleted by
an import. Detecting a meaning change is the audit's job; applying it is a human's.

The identity checks deserve one more sentence, because they land in two places. `PATIENT_ID`'s
`NotNull` sets the element's `Required` fact, which the schema writer persists on a shared
element-facts mixin - visible in the product like any other mixin value, but a stash rather than a
first-class metamodel slot, so the graph does not enforce it. The authoritative, machine-readable
copy of the check is the `cbioportalChecks` JSON above - the vendor-side record every import
refreshes.

## Type mapping in both directions

| cBioPortal datatype | Governed value type | Approximated |
|---|---|---|
| `NUMBER` | Double | yes - one type covers integers and decimals alike |
| `BOOLEAN` | Boolean | no |
| `STRING`, or an empty cell | String | no |
| anything else | String | yes |

Both approximations produce a lossiness record naming the path and the original string, and the
literal native type is preserved on the metadata mixin so audits compare `NUMBER` against `NUMBER`
rather than `Double` against `NUMBER`.

Generation inverts the map: Integer or Double becomes `NUMBER`, Boolean becomes `BOOLEAN`,
everything else becomes `STRING`. The inversion is not symmetric, and that is the honest headline
about round-tripping. A generated scaffold takes its display-name row from the governed Element
label, not from the `cbio.displayName` the import captured; it emits priority `1` for every column;
and it has no slot for identity checks, because a staging header block has nowhere to put them.
Descriptions do survive, read back from the metadata mixin, falling back to the label when absent.

## What the audit can and cannot say

The shared engine contributes coverage and drift codes; the connector contributes conformance rules.
For a cBioPortal estate the live set is:

* **Coverage** - `dataset-unmapped` (Warning): an instrument in the files is not governed.
  `field-unmapped` (Info): an attribute exists in the files but not on the governed Type.
* **Drift** - `field-removed` (Error): a governed attribute is gone from a submitted instrument.
  `field-type-drift` (Error): the datatype row moved away from what the last import recorded, with
  the comparison run on the raw strings (`governed: NUMBER, estate: STRING`), not on the approximated
  value types.
* **Conformance** - `attribute-no-description` (Info), raised once per instrument with the offending
  attribute names in the detail; `attribute-id-not-upper` (Warning), raised per attribute whose ID
  contains anything other than upper-case letters, digits, and underscores.

Four shared codes cannot meaningfully fire here, and knowing why is knowing the engine.
`projection-unmapped` needs semantic models or exposures, which staging files do not have.
`contract-drift` needs a contract-enforcement flag, which staging files do not have; the connector
always records `false`, so governed and estate always agree.

`dataset-removed` is the interesting one. The engine scopes removal by identity *namespace* -
everything before the last dot - so a project governing several estates of one vendor never reports
another estate's tables as deleted. A cBioPortal instrument identity, `patient` or `sample`, has no
dot in it, so each identity is its own namespace: an instrument you did not submit is out of scope
rather than removed, and the code never fires here. The payoff is that auditing only
`clinical_patient` is safe, which is what you want on a pull request that touched one file.

The `enum-*` codes need accepted values, and clinical staging files declare none - with one important
exception. If a human governs a clinical attribute with a taxonomy (a controlled list of `OS_STATUS`
values, say), every subsequent audit will report `enum-constraint-removed` (Warning), because the
estate side genuinely declares no value set. That is the engine telling the truth about an asymmetry,
not a false positive, but it is worth knowing before someone adds the taxonomy.

## Limits, and where they bite

* **Clinical files only.** Mutation, CNA, and expression files are data, not schema, and are out of
  scope by design.
* **No live portal connection.** The connector declares Import, Audit, and Generate - not LiveSync.
  CoreModels never holds portal credentials; every verb runs on artifacts you supply.
* **No lineage.** A study carries no build DAG, so `lineageEdgesAdded` is always zero. The
  patient/sample join is a reference, not a lineage edge.
* **The snapshot cap.** The parsed estate is stored gzipped and base64-encoded for re-audit, capped
  at roughly 1.5 million encoded characters. Past that, the import reports `snapshotStored: false`
  with a lossiness record: fresh-artifact audits still work, but re-audit has no baseline.
* **Additive re-import is conservative.** When a later import adds an attribute to an instrument
  that is already governed, the attribute is created - but reference semantics are not auto-created
  for it. The import says so with a `ConstraintRelaxation` record explaining that the facts are kept
  on the metadata mixin and surfaced by the audit, because changing governed meaning is a human act.

None of these are hidden: each surfaces as a lossiness record on the call that hit it, or as a finding
on the next audit. The connector reads exactly what a study's header block declares, flags every
approximation, and puts each fact where it belongs - which is what gives a drift audit something
precise to measure against.

For the artifact contract and the extraction recipe, see the cBioPortal quickstart in the CoreModels
integration docs.