# REDCap — Quickstart: REDCap to CoreModels: Zero to First Audit with One CSV

Most schema estates take real work to extract: information-schema queries, catalog exports, manifest builds. A REDCap project is the pleasant exception - the entire schema estate is one stable file, the data dictionary CSV, and you almost certainly already know how to download it. That makes the distance from "nothing governed" to "first drift audit" unusually short. This tutorial walks the whole distance: export the dictionary, import it into a governed CoreModels project, run your first audit, and read what comes back. No REDCap credentials are ever shared with CoreModels at any point.

# REDCap to CoreModels: Zero to First Audit with One CSV

Most schema estates take real work to extract: information-schema queries, catalog exports, manifest builds. A REDCap project is the pleasant exception - the entire schema estate is one stable file, the data dictionary CSV, and you almost certainly already know how to download it. That makes the distance from "nothing governed" to "first drift audit" unusually short. This tutorial walks the whole distance: export the dictionary, import it into a governed CoreModels project, run your first audit, and read what comes back. No REDCap credentials are ever shared with CoreModels at any point.

## What you need

- A CoreModels project and its 32-character hex project id. We'll call it `$PROJECT_ID`.
- A CoreModels login token, `$TOKEN`. Import requires the **Admin** role on the project; the audit only needs **Viewer**.
- Your CoreModels API base URL. We'll use `https://coremodels.example.com` throughout - substitute your own host.
- Access to your REDCap project (either the web UI or an API token you keep on your own machine).

## Step 1 - Export the data dictionary

The connector needs exactly one artifact, named `data_dictionary`. There are two ways to produce it.

**From the REDCap UI:** Project Setup → Data Dictionary → *Download the current Data Dictionary*. Save the CSV.

**From the REDCap API**, run locally - your REDCap token never leaves your machine and never reaches CoreModels:

```bash
curl -s -X POST "https://your-redcap.example.edu/api/" \
  -d token=$REDCAP_TOKEN -d content=metadata -d format=csv > data_dictionary.csv
```

Either route gives you the same file. The parser matches columns by header keyword rather than exact position, so lightly renamed exports still parse, and it reads RFC 4180 CSV properly - quoted choice lists with embedded commas are handled. Three columns must be present for the file to be recognized as a REDCap dictionary at all: *Variable / Field Name*, *Form Name*, and *Field Type*.

## Step 2 - Import it into the governed graph

The import call is a POST with the CSV contents inlined as a JSON string. The cleanest way to build that body is `jq --rawfile`, which handles all the escaping:

```bash
jq -n --rawfile dd data_dictionary.csv \
  '{artifacts: {data_dictionary: $dd}}' > import-request.json

curl -sS -X POST \
  "https://coremodels.example.com/graph/integrations/redcap/import/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  --data-binary @import-request.json
```

Import is **additive**: it creates governed nodes for what it finds and never mutates or deletes nodes that already exist. You can re-run it safely; already-governed instruments are skipped and counted, not overwritten. Applying a meaning change to the governed model is deliberately a human act - drift between the estate and the model is the audit's job, which is exactly what we set up next.

What the import writes, in CoreModels vocabulary:

- Each **instrument (form)** becomes a governed **Type**.
- Each **variable** becomes an **Element** on its Type, with the text validation type mapped to a real data type - `integer` → Integer, `number` → Double, `date_*` → DateTime, `yesno`/`truefalse` → Boolean, `calc` → Double.
- **Radio, dropdown, and checkbox choices** like `1, Male | 2, Female` become **Taxonomies**. The human labels govern; the numeric codes ride along as `redcap.codes` metadata so nothing is lost.
- `Required Field? = y` becomes a **NotNull** check; the first variable in the dictionary - REDCap's record id - is recognized as the instrument's identity and gets **Unique + NotNull**.
- Every field marked `Identifier? = y` (REDCap's PHI flag) is inventoried as `redcap.identifier` metadata. Branching logic is preserved as metadata too.
- Purely presentational `descriptive` rows are skipped - and counted, so you can verify nothing silently vanished.

## Reading the import response

```json
{ "success": true, "vendor": "redcap", "projectName": "redcap-project",
  "datasetsAdded": 3, "datasetsSkippedExisting": 0, "fieldsAdded": 0,
  "lineageEdgesAdded": 0, "lineageEdgesSkipped": 0, "nodesEnriched": 67,
  "snapshotStored": true, "lossiness": [], "errors": [] }
```

Read those counters precisely, because one of them is easy to misread. `datasetsAdded` is the number of instruments newly governed - three here. `fieldsAdded` is *not* your variable count: it counts variables added to instruments that were **already** governed, so on a first import it is legitimately `0` - the variables of a brand-new instrument are written together with their Type. The counter that reflects the whole estate is `nodesEnriched`: one write per instrument plus one per variable, so 3 + 64 = 67. `lineageEdgesAdded` stays 0 for REDCap; a data dictionary carries no lineage. Two more fields deserve attention:

- `lossiness` is the honesty channel. A successful import can still report exactly what it approximated or dropped - for example, a duplicate variable name in the dictionary is recorded here (the later row is ignored). Empty means a clean, faithful import.
- `snapshotStored: true` means the parsed dictionary was persisted server-side. That snapshot is what powers one-call re-audits later, so you'll want it to be true. Very large dictionaries that exceed the storage cap (~1.5 MB encoded) report `false` with a lossiness record explaining it - fresh-artifact audits still work fine.

## Step 3 - Run your first audit

The audit compares a data dictionary against the governed model and reports three sections: **Coverage** (what is and isn't governed), **Drift** (what changed since governance), and **Conformance** (research-data hygiene rules specific to REDCap). It is strictly read-only and needs only Viewer role. Setting `recordHistory: true` additionally appends this run to the project's rolling audit trail - useful from day one, because it seeds the status badge:

```bash
jq -n --rawfile dd data_dictionary.csv \
  '{artifacts: {data_dictionary: $dd}, recordHistory: true}' | \
curl -sS -X POST \
  "https://coremodels.example.com/graph/integrations/redcap/audit/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  --data-binary @- | jq '{errorCount, warningCount, infoCount, codes}'
```

Since you just imported this exact dictionary, expect zero drift. What you will likely see immediately are the REDCap conformance findings:

```json
{ "section": "Conformance", "severity": "Info", "code": "phi-fields",
  "subject": "demographics",
  "message": "Instrument carries 3 identifier (PHI) field(s) - inventory them before any export or agent access.",
  "detail": "first_name, last_name, mrn" }
```

`phi-fields` is the aggregated PHI inventory - one finding per instrument, naming the flagged fields. It's informational, and it's arguably the single most useful thing a first audit gives a research data manager: a machine-readable list of exactly which variables carry identifiers, per instrument, derivable by any downstream policy check. Two more REDCap-specific rules may appear: `text-no-validation` (Info - a free-text field with no validation type, which resists harmonization) and `duplicate-choice-codes` (Warning - a choice list reuses a code, making exported data ambiguous).

The response also carries three counters - `errorCount`, `warningCount`, `infoCount` - plus a `findings` array and a `markdown` field holding a human-readable report you can paste straight into a ticket or PR comment. The rule that matters most going forward: `errorCount > 0` means the dictionary violates governed meaning. That is the fail signal you'll later wire into CI.

## Step 4 - The badge and the status check

Because you recorded the audit run, the project now has a live status badge - a self-contained SVG reflecting the latest recorded run:

```bash
curl -sS -H "Authorization: Bearer $TOKEN" \
  "https://coremodels.example.com/graph/integrations/redcap/badge/$PROJECT_ID" \
  > redcap-audit.svg
```

Green means clean, yellow means warnings only, red means errors, gray means no recorded runs yet. And any time you want to confirm what's governed, the status endpoint returns the last-import state and the governed dataset count:

```bash
curl -sS -H "Authorization: Bearer $TOKEN" \
  "https://coremodels.example.com/graph/integrations/redcap/status/$PROJECT_ID"
```

## Where this goes next

You now have a governed model of your REDCap project and a repeatable, read-only check against it. From here the loop tightens in three directions: wire the audit into CI so every dictionary change is gated before it ships; use the one-call `reaudit` endpoint to check the stored dictionary snapshot whenever the *governed model* changes; and close the circle with `generate`, which emits an upload-ready data dictionary CSV back out of the governed model. Every governed fact - including that PHI inventory - is also queryable by AI agents over MCP.

For the condensed version of everything above, see the REDCap quickstart in the CoreModels docs (`docs/quickstarts/redcap`).