# Apache Avro — API: Avro Over HTTP: Four Endpoints and Their Honest Limits Every transform route in CoreModels answers with the same envelope, so learning the contract once buys you the whole surface: # Avro Over HTTP: Four Endpoints and Their Honest Limits Every transform route in CoreModels answers with the same envelope, so learning the contract once buys you the whole surface: ```jsonc { "success": true, // false only if it could NOT proceed - then read "errors" "lossiness": [ { "kind": "...", "path": "...", "explanation": "..." } ], "errors": [], // on failure: [{ "path": "...", "message": "..." }] "schema": "..." // or projectId / summary / data, depending on the route } ``` For Avro, four routes matter. This article walks all four with real bodies and responses, and states the limits where they exist rather than leaving you to trip over them. Start with direction, because we publish it per format. The key is `avro`, and it appears in both lists - decode (`jsonschema | shex | avro | jsonld | sql | osi | osi-json | owl | linkml | protobuf | odcs | odm`) and encode (`jsonschema | shex | avro | jsonld | sql | osi | osi-json | owl | linkml | protobuf | odcs | synapse`). Some formats are one-way - `odm` decodes only, `synapse` encodes only - but Avro round-trips. Its real constraints are structural: **the document root must be a `record`**, and an Avro document has exactly one root, so an export produces one record schema rather than a bundle. | # | Route | Role | Writes? | |---|---|---|---| | 1 | `POST graph/transform/schema/import/{projectId}` | Admin | yes | | 2 | `POST graph/transform/schema/export/{projectId}` | Viewer | no | | 3 | `POST graph/transform/schema/map/{projectId}` | Viewer (`ai`: Editor) | no | | 4 | `POST graph/transform/plan/execute/{projectId}` | Viewer | no | All four sit under `https://coremodels.example.com/graph/transform/...` (no `api/` prefix), take `Authorization: Bearer $TOKEN` and `Content-Type: application/json`, and are project-scoped like every other CoreModels graph API. ## 1. `schema/import` - an `.avsc` becomes a governed model The body is `{ "format": "avro", "schema": "" }`, plus an optional `spaces` array (omit it for the project's main space). Take a record that carries annotations, so the response has something to say: ```json { "type": "record", "name": "Claim", "namespace": "org.example.claims", "fields": [ { "name": "claimId", "type": "string", "x-sia-role": "identifier", "x-maps-to": { "acme": "https://data.acme.example/claims/id" } }, { "name": "filedOn", "type": { "type": "int", "logicalType": "date" } }, { "name": "status", "type": { "type": "enum", "name": "ClaimStatus", "symbols": ["open", "settled", "denied"] } } ] } ``` ```bash jq -n --rawfile s Claim.avsc '{ format: "avro", schema: $s }' \ | curl -sS -X POST \ "https://coremodels.example.com/graph/transform/schema/import/$PROJECT_ID" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ --data-binary @- ``` The project gains a Type `Claim` with three Elements: `claimId` (String, required - a plain Avro field always carries a value), `filedOn` (DateTime, courtesy of the `date` logical type), and `status`, which references a new Taxonomy `ClaimStatus` whose terms are the enum symbols in order. Element labels keep the original field names; internal node ids are alphanumeric, because CoreModels ids must be. The response is the envelope with `projectId` as its payload, and its ledger is the honest part. `x-maps-to` is carried - it lands as a mapsTo mixin value, the same carrier the rest of the stack reads - but `x-sia-role`, `x-sia-priority`, and `x-sia-instruction` have no CoreModels slot, so each is declared: ```json { "kind": "StructuralDrop", "path": "Element[Claim.claimId]", "explanation": "SIA 'role' has no CoreModels carrier; it was dropped." } ``` Import aggregates read-side and write-side lossiness into one list. On a schema with no annotations and no exotic constructs, that list is simply `[]`. ## 2. `schema/export` - a governed model becomes an `.avsc` The reverse needs only the format key. There are no Avro-specific options: `vendor` belongs to `sql` output, the `synapse*` keys to `synapse` output, and both are ignored here. ```bash curl -sS -X POST \ "https://coremodels.example.com/graph/transform/schema/export/$PROJECT_ID" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "format": "avro" }' ``` Because Avro is a JSON-shaped format, `schema` in the response is a JSON **object**, not a string. Three encoder rules govern what you get, and they hold everywhere Avro is the target: required elements emit bare types, optional elements emit `["null", T]` unions, and DateTime emits a `timestamp-millis` logical type unless the model remembers a more specific one. You can watch all three on the stateless route by mapping a JSON Schema into Avro - same encoder, visible input: ```json { "type": "record", "name": "Customer", "fields": [ { "name": "customer_key", "type": "long" }, { "name": "full_name", "type": "string" }, { "name": "signup_date", "type": ["null", { "type": "long", "logicalType": "timestamp-millis" }] }, { "name": "loyalty_points", "type": ["null", "long"] } ] } ``` That is the output for a `Customer` object type with two required properties, an optional `date-time`, and an optional integer. Integers emit as `long` when the model did not originate in Avro; when the source was decoded from Avro in the same call - as on the stateless mapping route in §3 - the original token (`int` versus `long`, `float` versus `double`, `bytes`) is preserved and re-emitted exactly. Now the limit worth planning around. An Avro document has one root record, so this route emits one record: the root type plus every named type reachable from it, nested inline at first use and referenced by name afterwards. Types in the project that nothing references are not part of that document, and no ledger entry announces their absence - so count your types. Two practical answers: scope the export with `spaces`, or use the project-level Avro export on the machine-to-machine surface, `GET v1/{projectId}/exportAvro?spaceId=…&nodeIds=…` (Viewer role, `ApiResponse` envelope with the payload under `data`), where `nodeIds` selects exactly the types you want. Agents reach the same export through the `export_avro` MCP tool. ## 3. `schema/map` - Avro through the engine, toward a target `schema/map` is the stateless engine surface: decode the source, produce a plan against a target hint, validate it through the universal gate, execute, encode. Nothing touches the project - every call is inherently a dry run - and the response adds `plan` to the envelope. The hint is what you map *toward*. Pass the source as its own hint for a pure format conversion, pass a different schema in any decodable format to map onto something real, or set `"useProjectAsTargetHint": true` to map toward the project's own schema (that variant reads the project; it still writes nothing). The `inferred` strategy requires one of the three: with no hint it declines - `The inference resolver requires a target IR to match against.` ```bash jq -n --rawfile s OrderEvent.avsc '{ sourceFormat: "avro", sourceSchema: $s, targetFormat: "sql", vendor: "postgres", targetHintFormat: "avro", targetHintSchema: $s, mapping: { kind: "inferred", caseInsensitive: true } }' \ | curl -sS -X POST \ "https://coremodels.example.com/graph/transform/schema/map/$PROJECT_ID" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ --data-binary @- > map-result.json ``` For an `OrderEvent` record with an id carrying `x-maps-to`, a `double`, a nullable string, an enum, a string array, and a `timestamp-millis` field, the `schema` string renders as: ```sql CREATE TABLE "OrderEvent" ( "orderId" VARCHAR(255) NOT NULL, "amount" NUMERIC NOT NULL, "couponCode" VARCHAR(255), "status" VARCHAR(255) NOT NULL, "lineItems" VARCHAR(255) NOT NULL, "placedAt" TIMESTAMP NOT NULL ); COMMENT ON COLUMN OrderEvent.orderId IS '{"x-maps-to":{"schema.org":"https://schema.org/orderNumber"}}'; ``` with one ledger entry: `ConstraintRelaxation` at `Element[OrderEvent.status]` - "Postgres has no inline enum; emitted as VARCHAR (the allowed-value constraint is not enforced)." Note what traveled and what did not. The nullable union became a nullable column; the logical type became `TIMESTAMP`; the semantic annotation survived as a column comment; the enum's four allowed values did not, and the ledger says so with the path. The array landed in a single column, which is the other line to check before you fan out to a warehouse. `mapping.kind` also accepts `explicit` (an authored mapping guide, passed as JSON text in `mapping.guide`) and `ai` (a server-side Claude proposal that passes the identical validation gate). `ai` requires Editor or Admin membership plus a server-configured Anthropic key, and it sends schema content to the Anthropic API server-side. ## 4. `plan/execute` - replay the stored plan The `plan` from the previous call is the point of the previous call. One wire quirk: `plan/execute` takes it **as a string**. ```bash jq '.plan' map-result.json > orderevent-plan.json jq -n --rawfile s OrderEvent.avsc --rawfile p orderevent-plan.json '{ sourceFormat: "avro", sourceSchema: $s, plan: $p, targetFormat: "sql", vendor: "postgres" }' \ | curl -sS -X POST \ "https://coremodels.example.com/graph/transform/plan/execute/$PROJECT_ID" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ --data-binary @- ``` The parsed plan goes through the same gate as every strategy - a stored plan earns no shortcut - then executes deterministically. Same plan plus same source gives the same output. The hint fields are optional on replay: the plan already names its targets. And if the source has drifted out from under the plan, the gate says so instead of guessing, naming the operation index and the id that no longer resolves. ## Records, not just schemas `avro` is also a *data* format on the record endpoints (`data/import`, `data/export`, `data/map`), where it means Avro's JSON encoding rather than plain JSON. Two differences matter: an optional field is either `null` or a one-key union object, and a date-time is the underlying epoch-milliseconds `long`. ```json [ { "orderId": "A-1001", "amount": 42.5, "couponCode": { "string": "SPRING" }, "status": "paid", "lineItems": ["sku-1", "sku-2"], "placedAt": 1719792000000 }, { "orderId": "A-1002", "amount": 9.99, "couponCode": null, "status": "placed", "lineItems": ["sku-9"], "placedAt": 1719878400000 } ] ``` Enum values are bare symbols, collections are JSON arrays. The data-format list is `json | csv | jsonld | sql | avro`. ## Errors, honestly shaped Failures arrive inside the envelope with a path and a message, not as bare 500s: - **Wrong format key** - `Unknown schema format 'avsc'.` followed by the accepted list, so the fix is in the message. - **Malformed payload** - `Could not parse the 'avro' schema: …` with the parser's own detail. - **Non-record root** - `The root of an Avro schema must be a record.` at path `$`. A bare enum or a primitive as the document root is an error, not an approximation: there is no Type to build. - **Avro output with no root type** - `The schema has no root type to encode as an Avro record.` This is the one to remember when Avro is the *target* of a mapping: a plan that renames the root record can leave the encoder with nothing to root on. Keep the record name across the mapping. - **Inferred mapping without a hint** - the message from §3. The transform API documentation collects all ten endpoints, their roles, and ready-to-paste bodies for every format in one place.