# CoreModels Docs > How to build with CoreModels: quickstarts, API and MCP references, automation recipes and deep dives for every connector. The case for each connector — the problem it fixes — lives on https://www.coremodels.io/guides. ## Documentation - [Start here](https://learn.coremodels.io/): Three ways into CoreModels, five minutes each. - [Grounding an assistant](https://learn.coremodels.io/agents/grounding.md): Look up before you make up. - [MCP server](https://learn.coremodels.io/agents/mcp.md): Endpoint, auth, limits and worked examples. - [Tool reference](https://learn.coremodels.io/agents/tools.md): Every MCP tool with signature and example. - [Connectors](https://learn.coremodels.io/connectors.md): Every documented connector. - [Plans and limits](https://learn.coremodels.io/account/plans): What each plan allows, with the live numbers. - [Recipes](https://learn.coremodels.io/recipes): The shipped recipes, one page each. - [Video series](https://learn.coremodels.io/videos): Seven episodes on meaning in data pipelines. - [Changelog](https://learn.coremodels.io/changelog.md): What shipped, newest first. - [Roles](https://learn.coremodels.io/account/roles.md): CoreModels provides the following roles, listed in order of authority. - [Neo AI Agent](https://learn.coremodels.io/agents/neo.md): Neo is AI-powered assistant for interacting with CoreModels. It enables natural language conversations to read and create schema data without needing to navigate the UI directly. - [Integration APIs](https://learn.coremodels.io/api.md): This page explains how to view and access the Integration APIs, it also provides some general guidelines to insure the best experience - [Reference](https://learn.coremodels.io/api/reference.md): This is the complete API reference for CoreModels Integration APIs Version 1. - [Core Concepts](https://learn.coremodels.io/concepts.md): Welcome to the Core Concepts page. - [Core Data Model](https://learn.coremodels.io/concepts/core-data-model.md): What is a data model? - [FAQ](https://learn.coremodels.io/faq.md): What's the default relation when creating Types and Elements ? - [Building your first Schema (Part 1)](https://learn.coremodels.io/getting-started/first-schema.md): Welcome to your first hands-on guide with CoreModels. - [Relations and Taxonomies (Part 2)](https://learn.coremodels.io/getting-started/relations-and-taxonomies.md): In this section we'll learn about the default relations, create another type Scientific Post, and a taxonomy for the StudyType. - [Crash Course](https://learn.coremodels.io/getting-started/crash-course.md): The Crash Course shares essential features and knowledge to help you start designing your Core Models. - [JSON-LD](https://learn.coremodels.io/guides/json-ld.md): CoreModels supports importing schemas using JSON-LD, which offers a convenient way to enrich JSON files with required schemas, including those found in schema.org. - [Dynamic Import (JSON-LD)](https://learn.coremodels.io/guides/json-ld/dynamic-import.md): JSON-LD Dynamic importer - [Export (JSON-LD)](https://learn.coremodels.io/guides/json-ld/export.md): In this guide you will see how to create a JSON-LD profile manually, or simply just how to export your imported schema. - [Merge (JSON-LD)](https://learn.coremodels.io/guides/json-ld/merge.md): This guide assumes that you have already imported your schema, or at least followed the Dynamic Import (JSON-LD) guide, and imported the sample schema, thus having an import profile ready. - [JSON-LD @type property](https://learn.coremodels.io/guides/json-ld/json-ld-type-property.md): CoreModels applies this translation to any import type and handles the '@type' property accordingly. - [JSON Schema](https://learn.coremodels.io/guides/json-schema.md): CoreModels makes it easy to work with JSON Schema, a powerful vocabulary for ensuring JSON data consistency and validation. - [Import (JSON Schema)](https://learn.coremodels.io/guides/json-schema/import.md): Expand the features tray, then from channels, click on 'Import JSON-Schema' to open the Import UI. - [Export (JSON Schema)](https://learn.coremodels.io/guides/json-schema/export.md): In this guide you will see how to how to export your imported schema. - [Synchronize (JSON Schema)](https://learn.coremodels.io/guides/json-schema/synchronize.md): This guide assumes that you have already imported your schema, or at least followed the Import (JSON Schema) guide, and imported the sample schema. - [Push To Github (JSON Schema)](https://learn.coremodels.io/guides/json-schema/push-to-github.md): This guide assumes that you already have a schema in your CoreModels space, See also - [The Schema Creation Playbook](https://learn.coremodels.io/knowledge-base/schema-creation-playbook.md): The process of creating a schema can be broken down into a series of logical steps. - [Semantic Concepts](https://learn.coremodels.io/knowledge-base/semantic-concepts.md): Semantic Concepts: Controlled List / Taxonomy / Ontology - [Airbyte connector](https://learn.coremodels.io/connectors/airbyte.md): Govern the catalog every sync trusts - before a silent type change lands in the warehouse. - [Airbyte — Quickstart: Zero to First Audit: Putting an Airbyte Catalog Under Governance](https://learn.coremodels.io/connectors/airbyte/quickstart.md): Everything in this tutorial happens in a shell. There is no agent to install in your Airbyte deployment, no warehouse connection to configure, and no credential of yours that ever reaches CoreModels. You export one JSON document that Airbyte already produces, upload it, and ask a question about it. - [Airbyte — API: Every Verb, Every Role: The Airbyte HTTP Surface](https://learn.coremodels.io/connectors/airbyte/api.md): One sentence in the controller sets the shape of everything below it: the vendor integration surface is **read-authority**. Import writes to the graph, and only additively. Audit and generate never write anything. Recording an audit run in the history is opt-in bookkeeping - except for re-audit, which always records its run because that is the whole point of the verb. - [Airbyte — MCP: Tool Calls, Not Screenshots: Running an Airbyte Audit from an AI Agent](https://learn.coremodels.io/connectors/airbyte/mcp.md): There are two ways to let an AI assistant help with an ingestion catalog. The first is to paste the catalog into a prompt and ask what looks risky - fast, ungrounded, unverifiable. The second is to give the assistant a tool that runs the real audit against your governed model and returns coded findings. This article is about the second one. - [Airbyte — Automation: A Drift Gate for Airbyte: CI, Badges, and the Heartbeat Between Runs](https://learn.coremodels.io/connectors/airbyte/automation.md): A dbt gate has an obvious trigger: someone opens a pull request, the project compiles, and the compiled artifact is the thing you audit. Airbyte has no build step. A catalog is *discovered*, not compiled, and the schema it describes belongs to a system nobody in the repository controls. So the first question for an Airbyte drift gate is not "which endpoint do I call" - it is "where does `catalog.json` come from, and when". - [Airbyte — Deep dive: Inside the Airbyte Connector: Identity, Types, Checks, and the Limits We Publish](https://learn.coremodels.io/connectors/airbyte/deep-dive.md): Every vendor CoreModels integrates parses into one neutral shape: datasets with fields and normalized checks, plus lineage between them. Vendor detail rides in metadata bags, never as new top-level concepts, and connectors are pure - they parse, map types and contribute audit rules, never touching the graph. - [Apache Airflow connector](https://learn.coremodels.io/connectors/airflow.md): Govern the scheduler that already knows your freshness, ownership, and lineage. - [Apache Airflow — Quickstart: Zero to First Audit: Bringing an Apache Airflow Deployment under CoreModels Governance](https://learn.coremodels.io/connectors/airflow/quickstart.md): Your Airflow deployment can already describe itself. The stable REST API v1 will tell you every DAG, every task, and - if you use data-aware scheduling - every asset your pipelines produce and consume. What it will not tell you is whether any of that still matches what your organization *thinks* it runs: who owns each pipeline, which assets are consumed but produced by nothing, which consumers are quietly running on stale data because every producer is paused. - [Apache Airflow — API: The Airflow Integration API, End to End: Every Route, Role, and Payload](https://learn.coremodels.io/connectors/airflow/api.md): Every vendor integration in CoreModels answers the same first question the same way. Ask the platform what it knows how to govern: - [Apache Airflow — MCP: Governing Airflow with an AI Agent: The CoreModels MCP Tools in Practice](https://learn.coremodels.io/connectors/airflow/mcp.md): "Which of our pipelines have no accountable owner, and is anything downstream of a paused DAG?" That is a governance question, and until recently answering it meant a human clicking through the Airflow UI and cross-referencing a wiki. With CoreModels, an AI agent answers it directly: the same import/audit machinery we expose over HTTP is exposed over the Model Context Protocol, so an agent connected to your CoreModels server can inspect connector capabilities, run a full drift-and-hygiene audit of an Airflow deployment, and read back a human-quality report - all through typed tool calls. - [Apache Airflow — Automation: Catch Pipeline Drift Before It Ships: A CI Gate and Drift Loop for Airflow](https://learn.coremodels.io/connectors/airflow/automation.md): Orchestration drift is the quiet kind of failure. Nobody deletes a pipeline; a schedule is flipped to manual during an incident and never flipped back, and for three weeks a downstream table renders yesterday's world with perfect confidence. Nobody removes a dataset from governance; a DAG is renamed in a refactor and the governed model now describes a pipeline that no longer exists. None of this throws an exception - which is exactly why it belongs in CI, where a machine checks it on every change. - [Apache Airflow — Deep dive: Lineage First: How CoreModels Maps an Airflow Estate into the Governed Graph](https://learn.coremodels.io/connectors/airflow/deep-dive.md): Airflow is not a schema estate. A warehouse table has columns with types; a dbt model has a contract; an Avro subject has fields. A DAG has none of those - it has tasks, a schedule, an owner, and (with data-aware scheduling) declared relationships to the data it produces and consumes. So when we built the Apache Airflow connector for CoreModels, the design question was not "how do we pretend DAGs are tables?" but "what is orchestration's actual governance value?" The answer we committed to: the DAG-to-asset dependency graph, and pipeline hygiene. This connector is lineage-first, and every mapping decision below follows from that. - [Apache Avro connector](https://learn.coremodels.io/connectors/avro.md): One governed reading of the schema that currently lives in four places. - [Apache Avro — Quickstart: Your First Avro Transform: One Call, Three Things to Read](https://learn.coremodels.io/connectors/avro/quickstart.md): You have an `.avsc` file. Somewhere downstream, somebody needs the same shape as JSON Schema, or as a table, or as a data contract - and they need to know what the conversion cost. This article gets you from that file to a converted schema in a single HTTP call, and then teaches you to read the three parts of the answer: the schema, the plan, and the lossiness ledger. - [Apache Avro — API: Avro Over HTTP: Four Endpoints and Their Honest Limits](https://learn.coremodels.io/connectors/avro/api.md): Every transform route in CoreModels answers with the same envelope, so learning the contract once buys you the whole surface: - [Apache Avro — MCP: Handing Avro to an Agent: `transform_schema` End to End](https://learn.coremodels.io/connectors/avro/mcp.md): Ask a language model to "convert this Avro schema to LinkML" and it will improvise - plausible YAML on a good day, invented field names on a bad one. Connect it to CoreModels over MCP and the same sentence becomes a tool call: typed arguments, a deterministic engine, and a machine-readable account of what the conversion cost. This article is the whole loop for Avro - connection, exact arguments, a real conversion, and what a well-behaved agent does with the answer. - [Apache Avro — Automation: Ship the Plan, Not the Script: Avro Pipelines That Replay](https://learn.coremodels.io/connectors/avro/automation.md): Here is a job every streaming platform eventually has. An internal topic carries a rich record; a partner, a public catalog, or another business unit gets a redacted projection of it. Somebody writes a script. The script knows which fields to drop and which to rename, and that knowledge lives nowhere else. Six months later the source schema gains a field, the script silently passes it through, and the first person to notice is on the other side of the boundary. - [Apache Avro — Deep dive: Every Avro Construct, and Where It Lands in the IR](https://learn.coremodels.io/connectors/avro/deep-dive.md): Does an `int` come back as an `int`? - [AWS Glue connector](https://learn.coremodels.io/connectors/glue.md): Stop treating a crawler snapshot as the meaning of your lake. - [AWS Glue — Quickstart: Ten Minutes to a Governed Glue Catalog](https://learn.coremodels.io/connectors/glue/quickstart.md): Open a terminal. Everything in this tutorial is one AWS CLI command and two HTTP calls, and at the end of it your AWS Glue Data Catalog has a governed twin in CoreModels plus a first drift audit you can read line by line. Nothing gets installed in your AWS account, no IAM role is granted to us, and no credential of yours travels anywhere: you run the CLI, you upload the JSON it printed. - [AWS Glue — API: The AWS Glue Integration API, Route by Route](https://learn.coremodels.io/connectors/glue/api.md): CoreModels models vendor governance as a small, fixed set of verbs and applies them uniformly to every connector. For AWS Glue Data Catalog - vendor key `glue` - that is eight core routes on the interactive surface and two on the machine-to-machine surface. (A newer sync-plan surface - `sync/propose`, plan fetch, and the sync ledger - sits beside these and is not covered here.) This is the reference: the exact paths, the role each one enforces, the request bodies as the API defines them, and the responses you actually get back. - [AWS Glue — MCP: Handing the Glue Catalog to an Agent: CoreModels over MCP](https://learn.coremodels.io/connectors/glue/mcp.md): An agent that can answer *"did our lake drift?"* needs three things: a way to reach the governed model, a way to hand it a fresh catalog export, and a guarantee that asking the question cannot change the answer. The CoreModels MCP server provides all three. The same vendor-integration engine behind the HTTP routes is exposed as MCP tools, with the read verbs at Viewer role and the write verbs behind an admin endpoint and real project membership. - [AWS Glue — Automation: Automating Glue Drift: The Gate, the Trail, and the Heartbeat](https://learn.coremodels.io/connectors/glue/automation.md): Nothing in your repository changes when a crawler retypes a column. That is the awkward fact about lake governance: the estate you need to watch does not live in git, so the usual "run it on pull requests" reflex leaves the interesting drift entirely unobserved. A Glue drift gate has to be driven by a clock, not by a commit - and once it is, three more mechanisms fall out of it almost for free: a rolling trail, a status badge, and a server-side heartbeat that watches the *other* direction of drift. - [AWS Glue — Deep dive: What Happens to a Glue Table on Its Way into the Graph](https://learn.coremodels.io/connectors/glue/deep-dive.md): Take one entry out of an `aws glue get-tables` response - a table called `events` in database `lake`, a few columns, one partition key, a `classification` parameter, an S3 location - and follow it into CoreModels. By the end it is a governed Type with Elements, a vendor identity, a metadata mixin, an entry in an estate snapshot, and a set of audit rules watching it. This is that journey, plus the honest edges: what is approximated, what is deliberately not invented, and which behaviors will surprise you. - [Azure Synapse connector](https://learn.coremodels.io/connectors/synapse.md): Record the constraints that die on the way to a registered schema. - [Azure Synapse — Quickstart: From JSON Schema to a Registration-Ready Synapse Schema in One Call](https://learn.coremodels.io/connectors/synapse/quickstart.md): You have a JSON Schema. Synapse - Sage Bionetworks' data platform - will not accept all of it. Registered Synapse schemas are draft-07, and not even all of draft-07: the platform's `JsonSchema` REST object models a specific keyword subset, and anything outside it simply is not part of a registered schema. This quickstart takes a small, real schema across that boundary in one HTTP call to CoreModels (by ARAMAI), then reads the machine-readable ledger of exactly what the boundary cost. - [Azure Synapse — API: The Synapse Export, Route by Route: A CoreModels API Reference](https://learn.coremodels.io/connectors/synapse/api.md): Every format in the CoreModels transform surface belongs to two lists - the formats we decode and the formats we encode - and `synapse` appears in exactly one of them. The decode list is `jsonschema | shex | avro | jsonld | sql | osi | osi-json | owl | linkml | protobuf | odcs | odm`; the encode list swaps `odm` out and `synapse` in. This page is the working reference for that one direction: which routes produce a Synapse-ready schema, what each accepts, which one controls the registered-schema `$id`, and how to bring a Synapse schema back in anyway. - [Azure Synapse — MCP: Synapse Schemas from an Agent: transform_schema over MCP, End to End](https://learn.coremodels.io/connectors/synapse/mcp.md): Connect an MCP client - Claude Desktop, an IDE agent, your own orchestration - to a CoreModels deployment at `https://coremodels.example.com/mcp` (OAuth-protected), and two tools give it the entire Synapse workflow: `transform_schema` produces the registration-ready draft-07 schema, and `generate_synapse_manifests` produces the tabular curation companions. This article is the end-to-end session, with exact arguments and exact results, so an agent - or the person supervising one - knows precisely what each call does and does not do. - [Azure Synapse — Automation: Automating Synapse Schema Releases: Determinism, Plan Replay, and Version Discipline](https://learn.coremodels.io/connectors/synapse/automation.md): Run the same Synapse export twice against the same model and you get the same bytes. That single property - deterministic encoding - is what turns the CoreModels `synapse` profile from a converter into release machinery: outputs you can commit and diff, plans you can store and replay, and a version rule you can enforce mechanically instead of remembering. This article builds that pipeline piece by piece, with the honest constraints stated where they bite. - [Azure Synapse — Deep dive: Inside the synapse Coder: The Whitelist, the Ledger, and the Edge Cases](https://learn.coremodels.io/connectors/synapse/deep-dive.md): The entire CoreModels `synapse` profile can be stated in one sentence: *encode the neutral model as JSON Schema, then keep only what the Synapse `JsonSchema` REST object has a field for - and write down every single thing that rule removes.* This deep dive unpacks that sentence into the parts an integrator eventually needs: the IR-to-output mapping, the exact keyword whitelist and how strips are classified, the extras channel that carries verbatim keywords through the neutral model, the complete inventory of lossiness records the encoder can emit, and the edge cases our test suite pins - including the ones where we deliberately refuse to guess. - [cBioPortal connector](https://learn.coremodels.io/connectors/cbioportal.md): Govern the four comment rows that are the schema of your study. - [cBioPortal — Quickstart: Zero to first audit: putting a cBioPortal study's clinical schema under governance](https://learn.coremodels.io/connectors/cbioportal/quickstart.md): The clinical schema of a cBioPortal study does not live in a database, a catalog, or an API. It lives in the first five lines of two tab-separated text files that a curator maintains by hand. Those five lines decide whether an attribute passes portal validation, what it is called in the UI, whether it is a number or a string, and how prominently it is displayed. They are also, in most study repositories, the least reviewed lines in the project. - [cBioPortal — API: Every cBioPortal route, every role: the CoreModels HTTP surface for study governance](https://learn.coremodels.io/connectors/cbioportal/api.md): Ask the API what it can do for cBioPortal before you write a line of client code: - [cBioPortal — MCP: Handing a cBioPortal study to an agent: governance over MCP](https://learn.coremodels.io/connectors/cbioportal/mcp.md): "Check whether the study files on this branch still match what we govern, and tell me what changed." - [cBioPortal — Automation: One character, one broken study: automating cBioPortal drift checks in CI](https://learn.coremodels.io/connectors/cbioportal/automation.md): Here is a pull request diff from a study repository: - [cBioPortal — Deep dive: Following one clinical attribute all the way into the graph](https://learn.coremodels.io/connectors/cbioportal/deep-dive.md): Take a single column out of a cBioPortal patient staging file: - [Confluent connector](https://learn.coremodels.io/connectors/confluent.md): Put meaning under every Kafka subject the Schema Registry already inventories. - [Confluent — Quickstart: Confluent Schema Registry to CoreModels: Zero to First Audit](https://learn.coremodels.io/connectors/confluent/quickstart.md): Your Schema Registry already knows every event contract in your streaming platform - every subject, every version, every Avro record your producers have ever registered. What it does not know is what those contracts *mean*, whether the meaning is still what your consumers agreed to, and who is guarding it. In this tutorial we take a real registry from nothing to a completed schema audit in CoreModels: export the subjects with one shell loop, import the estate into a governed project, run the first audit, and read the result line by line. - [Confluent — API: The Complete HTTP Surface for Confluent Schema Registry Governance](https://learn.coremodels.io/connectors/confluent/api.md): This is the reference we wish every integration shipped with: every route, every role, every payload - nothing implied. CoreModels exposes eight core verbs for a Schema Registry estate across two HTTP surfaces, and this article walks all of them with real request and response bodies. The vendor key is `confluent`; the connector declares **Import, Audit, Generate**, so every verb below is genuinely available - including artifact generation, which not all of our connectors support. - [Confluent — MCP: Ask Your Agent Whether the Kafka Schemas Drifted: Confluent Governance over MCP](https://learn.coremodels.io/connectors/confluent/mcp.md): "Did anything in the registry drift from what we agreed?" is a question a platform engineer answers with three terminal commands and a diff. It is also, increasingly, a question they type into a chat window - and the agent on the other end needs real tools, not guesses. CoreModels ships its Confluent Schema Registry governance as a set of MCP tools, so any MCP-capable agent - Claude, Claude Code, or anything speaking streamable HTTP - can run the same import, audit, generate, and status verbs the HTTP API exposes, with the same role enforcement and the same read-only guarantees. - [Confluent — Automation: Failing the Build Before the Consumer Breaks: A CI Drift Gate for Confluent Schema Registry](https://learn.coremodels.io/connectors/confluent/automation.md): The Schema Registry's compatibility checks are necessary and not sufficient. They guarantee a new schema version can be deserialized by existing consumers - wire compatibility. They say nothing about whether `amount` is still the governed Double your downstream jobs assume, whether an enum quietly grew a symbol nobody reviewed, or whether a topic your reports depend on disappeared from the registry altogether. Those are questions about *meaning*, and meaning drift in streaming breaks things at runtime, in production, at consumer speed. This article wires the CoreModels schema audit into CI as a merge gate for schema changes, then completes the loop with the status badge, the rolling history, one-call re-audits, and the scheduled heartbeat. - [Confluent — Deep dive: Anatomy of a Governed Streaming Estate: How the Confluent Connector Maps Subjects into the Graph](https://learn.coremodels.io/connectors/confluent/deep-dive.md): A schema registry is three naming systems wearing one trench coat. There is the *subject* (`orders-value`), the registry's versioned unit. There is the *record* (`com.acme.Order`), the Avro type the schema declares. And there is the *topic* (`orders`), the physical stream the subject-name strategy encodes. Most tooling collapses these into one string and loses information doing it. This deep dive walks exactly how CoreModels maps a Confluent Schema Registry export into the governed graph - what becomes a Type, an Element, a Taxonomy or a reference; what rides metadata; which audit rules are this vendor's; and where the mapping is lossy, because a governance product that hides its own approximations cannot be trusted about anyone else's. - [Databricks connector](https://learn.coremodels.io/connectors/databricks.md): Give Unity Catalog something to check thousands of columns against. - [Databricks — Quickstart: Zero to First Audit: Governing a Databricks Unity Catalog Estate with CoreModels](https://learn.coremodels.io/connectors/databricks/quickstart.md): This is a working session, not a tour. By the end of it you will have a governed model of a Unity Catalog schema inside CoreModels, and you will have run your first drift audit against it - without installing anything in your workspace and without handing us a single credential. Everything CoreModels learns about your lakehouse arrives as JSON files you extract yourself from the Databricks SQL editor. - [Databricks — API: Every Endpoint of the CoreModels Databricks Integration, With Real Payloads](https://learn.coremodels.io/connectors/databricks/api.md): Reference articles usually show the happy path and gesture at the rest. This one is the full inventory: every HTTP route the CoreModels Databricks Unity Catalog connector answers on, the role each one enforces, the request and response bodies as they actually are, and - where a verb lives on one surface but not the other - an explicit statement of that fact rather than a diagram that implies otherwise. - [Databricks — MCP: Ask Your Agent Whether the Lakehouse Drifted: Unity Catalog Governance over MCP](https://learn.coremodels.io/connectors/databricks/mcp.md): "Has our Unity Catalog estate drifted from what we agreed it means?" is a question an AI agent can now answer with evidence instead of vibes. The same governance verbs that back the CoreModels HTTP API - status, audit, import, generate - are exposed as tools on our MCP server, so an agent can inspect a governed Databricks estate, audit a fresh extract against it, and draft the fix, all inside one conversation. This article walks that loop exactly as an agent drives it, with the real tool names and arguments, including the `artifactUrls` path for extracts too large to paste into a chat. - [Databricks — Automation: Fail the Build, Not the Dashboard: A CI Drift Gate for Databricks Unity Catalog](https://learn.coremodels.io/connectors/databricks/automation.md): Somewhere in your lakehouse, a notebook is about to rebuild a table with a widened column type, and nothing will fail until a dashboard does - days later, in front of the wrong audience. The cheapest place to catch that is a pull request. This article turns the CoreModels schema audit into a merge-blocking CI gate for a Unity Catalog estate, then builds out the rest of the operational loop most drift setups forget: the status badge, the rolling history, the one-call re-audit for when the governed model itself changes, and the scheduled heartbeat that keeps watch between releases. - [Databricks — Deep dive: Anatomy of an Import: How a Unity Catalog Estate Becomes a Governed Graph](https://learn.coremodels.io/connectors/databricks/deep-dive.md): Take a single row of a Unity Catalog extract: - [dbt connector](https://learn.coremodels.io/connectors/dbt.md): Meaning has no home in dbt. Bring a manifest; we turn `accepted_values` into a vocabulary you can govern and your agent can read. - [dbt — Quickstart: Manifest In, Contracts Out: A dbt Round Trip in Five Steps](https://learn.coremodels.io/connectors/dbt/quickstart.md): Most integrations treat dbt as somewhere to read from. This one also publishes back: governed meaning leaves CoreModels as dbt model property files with enforced contracts, one file per model, colocated beside that model's own `.sql`. This walkthrough runs the whole loop — artifacts out of dbt, meaning governed on top of them, contracts back into the repo, `dbt build` green. Nothing here needs a warehouse credential or a dbt platform connection: artifacts in, artifacts out. CoreModels never writes to your repo and never opens a pull request. Generate returns files; your own PR flow lands them. - [dbt — API: Four Verbs and a Fan-Out: The dbt Integration over HTTP](https://learn.coremodels.io/connectors/dbt/api.md): The dbt connector declares Import, Audit, and Generate, and four routes carry the loop that matters: push artifacts in, check them against governed meaning, publish contracts back out, and ask what the last import knew. This is the route-by-route reference — role gating, request bodies field by field, exact response shapes. Throughout, `https://coremodels.example.com` stands in for your deployment, `$TOKEN` for your bearer token, and `$PROJECT_ID` for the 32-character hex id of the governing project. The vendor key is `dbt`. One property runs through all four: no credentials — artifacts in, artifacts out. CoreModels never connects to your warehouse, never runs dbt, never reads a dbt platform API, and never writes into your repository; Generate hands back files, and your own PR flow lands them. - [dbt — MCP: Four Tools and a Ledger: dbt Contracts from an Agent's Seat](https://learn.coremodels.io/connectors/dbt/mcp.md): "Generate the contracts for `stg_orders` and `stg_customers`, and tell me if anything won't apply cleanly." That sentence is an afternoon of dbt property-file maintenance, and an agent connected to CoreModels over MCP can answer it with real tool calls instead of plausible-looking YAML. Four vendor integration tools give the agent the same governance surface a human gets over HTTP: the same role checks, the same read-only guarantees, and the same honest ledger of what could not be represented. - [dbt — Automation: Two Loops and an Empty Diff: dbt Contract Automation in CI](https://learn.coremodels.io/connectors/dbt/automation.md): Try this on a branch first, because it reframes what the gate is for. Take a model with an `accepted_values` test, delete the test, and open a pull request. - [dbt — Deep dive: Every Line Has a Source: Inside the dbt Contract Generator](https://learn.coremodels.io/connectors/dbt/deep-dive.md): CoreModels publishes governed meaning *into* a dbt project. You call Generate and get back dbt model property files with enforced contracts — one per model, colocated with the model's own `.sql` — which your own pull request flow lands in the repo. CoreModels never writes to your repo, never opens the PR, never runs dbt, and never connects to a warehouse; it returns files. Which means the files have to be checkable. Every line the generator emits traces back to a fact recorded in the governed model, and where a fact is missing it omits the line and says why rather than guessing. This is the mechanism in the order it runs, so an engineer reading the output can verify it against the rules instead of trusting it. - [Google BigQuery connector](https://learn.coremodels.io/connectors/bigquery.md): Turn INFORMATION_SCHEMA from an inventory into a contract you can audit. - [Google BigQuery — Quickstart: Your First BigQuery Drift Audit with CoreModels](https://learn.coremodels.io/connectors/bigquery/quickstart.md): *Zero to a recorded schema audit of a BigQuery dataset: one SQL query, two HTTP calls, no credentials shared.* - [Google BigQuery — API: The CoreModels BigQuery Integration, Route by Route](https://learn.coremodels.io/connectors/bigquery/api.md): *The complete HTTP surface for governing a BigQuery estate: every verb, every role, every payload.* - [Google BigQuery — MCP: Governing BigQuery from an AI Agent over MCP](https://learn.coremodels.io/connectors/bigquery/mcp.md): *The same import–audit–generate loop, driven by tool calls instead of curl - including how agents ship multi-megabyte extracts.* - [Google BigQuery — Automation: A CI Drift Gate for BigQuery: Fail the PR, Not the Dashboard](https://learn.coremodels.io/connectors/bigquery/automation.md): *Turning the CoreModels schema audit into an automated gate - the v1 API, `errorCount` semantics, the badge, the history trail, and the two directions drift can come from.* - [Google BigQuery — Deep dive: Inside the BigQuery Connector: How an Estate Becomes a Governed Graph](https://learn.coremodels.io/connectors/bigquery/deep-dive.md): *What actually happens between a flat JSON extract and a queryable governed model - identities, type mappings, metadata mixins, audit rules, and the approximations we admit to.* - [JSON Schema connector](https://learn.coremodels.io/connectors/jsonschema.md): Stop asking a validator four questions it was only built to answer one of. - [JSON Schema — Quickstart: One Call, Three Answers: Your First JSON Schema Transform](https://learn.coremodels.io/connectors/jsonschema/quickstart.md): Every transform call in CoreModels returns three things, and the third one is the reason we built it this way. You get the converted schema. You get the plan that produced it. And you get a ledger of everything the conversion could not carry across exactly - written in English, with a path pointing at the construct it happened to. - [JSON Schema — API: Four Routes and a Round Trip: JSON Schema Over HTTP](https://learn.coremodels.io/connectors/jsonschema/api.md): A couple of format keys in the CoreModels transform surface are honest about being one-way. `odm` decodes only, because entity documentation is authored, not generated. `synapse` encodes only, and says so out loud when you try the other direction: *"'synapse' is encode-only: a Synapse schema is plain draft-07 JSON Schema - decode it with the 'jsonschema' format."* - [JSON Schema — MCP: Give an Agent a JSON Schema and a Target: transform_schema Over MCP](https://learn.coremodels.io/connectors/jsonschema/mcp.md): An agent working on your repository will find a `.schema.json` file long before it finds your conversion documentation. What happens next depends entirely on the tools it can reach. Without one, it writes a converter, or worse, writes the target schema from scratch and presents its guesses with the same confidence as facts. - [JSON Schema — Automation: Same Plan, Same Bytes: Automating JSON Schema Conversion](https://learn.coremodels.io/connectors/jsonschema/automation.md): A generator you cannot re-run and get identical output from is not a build step. It is a rumor with a timestamp. That is the practical objection to putting schema conversion in a pipeline: if today's run can differ from yesterday's for reasons nobody logged, then the generated Avro, the generated DDL, and the generated proto files are all provisional, and reviewing them is theater. - [JSON Schema — Deep dive: The Fidelity Contract: JSON Schema to IR and Back, Construct by Construct](https://learn.coremodels.io/connectors/jsonschema/deep-dive.md): Every format coder in CoreModels has to answer the same awkward question: what do you do with the parts of a document you do not model? There are three bad answers - drop them, guess at them, or refuse the document - and one good one, which is to carry them untouched and be explicit about the handful of cases where carrying them is not enough. - [JSON-LD connector](https://learn.coremodels.io/connectors/jsonld.md): Make the vocabulary you already trust executable downstream. - [JSON-LD — Quickstart: Zero to First Transform: Running a JSON-LD Vocabulary Through CoreModels](https://learn.coremodels.io/connectors/jsonld/quickstart.md): By the end of this page you will have made exactly one HTTP call and produced two artifacts: a JSON Schema generated from an RDF vocabulary, and a lossiness ledger that states - in writing - what the conversion could not carry across. Learning to read both is the entire skill. Everything else in the CoreModels transform surface is a variation on this loop. - [JSON-LD — API: Four Verbs and a Vocabulary: The JSON-LD HTTP Surface, With Real Bodies](https://learn.coremodels.io/connectors/jsonld/api.md): Everything you can do with an RDF vocabulary on the CoreModels HTTP API fits in four routes. One writes a vocabulary into a governed project, one publishes a project back out as a vocabulary, one converts statelessly through the mapping engine, and one replays a stored conversion. This article walks all four with request and response bodies we actually ran, then states the direction limits plainly - including the one that has no ledger entry to warn you about it. - [JSON-LD — MCP: Three Questions an Agent Must Answer After Converting a Schema](https://learn.coremodels.io/connectors/jsonld/mcp.md): When an agent converts a vocabulary into something else, it should be able to answer three questions afterwards: what did it produce, what did it lose, and can it do that again identically? Most conversion tooling answers the first. The `transform_schema` tool on the CoreModels MCP server answers all three in a single call - the produced schema, an explicit lossiness ledger, and the executed plan as a replayable artifact. This article runs a JSON-LD vocabulary through it end to end, with the exact arguments and the exact response. - [JSON-LD — Automation: Boring on Purpose: JSON-LD Conversion Pipelines That Produce the Same Bytes Every Time](https://learn.coremodels.io/connectors/jsonld/automation.md): A generated artifact is only trustworthy if it is boring. If regenerating last week's JSON Schema from the same vocabulary reshuffles keys, renames anonymous constructs, or quietly drops a constraint, then the diff in your pull request is noise and nobody reads it. This article is our recipe for the opposite: JSON-LD conversions that are byte-stable, gated by a reviewed plan, and loud when meaning is lost. - [JSON-LD — Deep dive: The Coder That Never Complains: A Full Accounting of JSON-LD Fidelity in CoreModels](https://learn.coremodels.io/connectors/jsonld/deep-dive.md): Here is the fact that should make you suspicious: in either direction, the CoreModels JSON-LD coder emits no lossiness records at all. Not "few" - none. Most coders in the engine keep a list of things they apologize for; this one's ledger is empty. That is either an honest structural property or a place where losses hide, so this article is the accounting: what maps to what, what rides in the extras channel, what the `@id` contract guarantees, where fidelity breaks, and what the tests pin down. - [LinkML connector](https://learn.coremodels.io/connectors/linkml.md): Let the pipeline read the model that is already right. - [LinkML — Quickstart: LinkML to Postgres in One Call: A CoreModels Quickstart](https://learn.coremodels.io/connectors/linkml/quickstart.md): Ten minutes, one YAML file, one HTTP call. By the end of this you will have converted a LinkML schema into Postgres DDL through CoreModels, read the machine-readable ledger of everything the conversion could not carry exactly, and sent the same schema back out as LinkML unchanged. - [LinkML — API: The LinkML HTTP Surface: Import, Export, Map, Replay](https://learn.coremodels.io/connectors/linkml/api.md): There are ten routes on the CoreModels transform API. Five of them matter if LinkML is your source or your target, and the first thing to settle is which one you actually want - because the difference between them is not the format, it is whether anything gets written and who decides what passes through. - [LinkML — MCP: Give an Agent a LinkML Schema: `transform_schema` over MCP](https://learn.coremodels.io/connectors/linkml/mcp.md): > **Analyst:** Here's our product catalog in LinkML. The warehouse team needs a Postgres table for it. What do we lose? - [LinkML — Automation: Plans, Not Scripts: Repeatable LinkML Conversion Pipelines](https://learn.coremodels.io/connectors/linkml/automation.md): A conversion you cannot reproduce is not a pipeline; it is a favor someone did once. The interesting question for automation is not "can this tool turn LinkML into Postgres DDL" - it is "will the run in six months, on a build agent, with a token nobody remembers issuing, produce the same bytes and tell me if it did not." - [LinkML — Deep dive: Inside the LinkML Coder: Every Construct, Every Loss](https://learn.coremodels.io/connectors/linkml/deep-dive.md): Start with one line of YAML: - [MACH ODM connector](https://learn.coremodels.io/connectors/odm.md): Stop shipping a recollection of the standard as if it were the standard. - [MACH ODM — Quickstart: Your First MACH ODM Transform: Markdown In, JSON Schema Out](https://learn.coremodels.io/connectors/odm/quickstart.md): The MACH Alliance publishes its Open Data Model (ODM) as entity documentation: Markdown files with prose, a field table, embedded YAML schema definitions, and sample objects. That is a great format for humans and a frustrating one for machines - you cannot validate a payload against a Markdown page. - [MACH ODM — API: MACH ODM over HTTP: One Converter, Two Importers, and an Honest "No Export"](https://learn.coremodels.io/connectors/odm/api.md): Every schema format on the CoreModels transform surface declares its direction, and we hold ourselves to those declarations in public. The `odm` format - MACH Alliance Open Data Model entity documents - is **decode-only**: ODM entities are authored documentation, not a generated artifact, so there is no encode back to prose. Rather than paper over that, the HTTP surface is built around it. This article walks the complete set of routes that touch ODM, with real request and response bodies, the role each route requires, and exactly what happens when you try the direction that does not exist. - [MACH ODM — MCP: Converting MACH ODM Entities with an Agent: transform_schema over MCP](https://learn.coremodels.io/connectors/odm/mcp.md): Picture the request as it actually arrives: someone pastes a MACH Alliance Open Data Model entity document into a chat and asks their agent, "stand up a Postgres table for this." The document is Markdown - an H1, some prose, YAML blocks inside a schema section. Between that paste and a `CREATE TABLE` statement sits exactly one tool call. - [MACH ODM — Automation: ODM at Scale: Batch Conversion, Deterministic Pipelines, and Replayable Plans](https://learn.coremodels.io/connectors/odm/automation.md): One entity document is a demo. A standards repository is dozens of them - identity, product, inventory, pricing - revised by pull request, consumed by teams who need formal schemas, not Markdown. The moment MACH Open Data Model documents become an input to your build, three engineering questions appear: can the conversion run unattended, will the same input always produce the same output, and what happens when one file in the batch is broken? - [MACH ODM — Deep dive: Inside the MACH ODM Coder: What Maps, What Rides Along, What Gets Reported](https://learn.coremodels.io/connectors/odm/deep-dive.md): A MACH Alliance Open Data Model entity document is five things wearing one Markdown file: an H1 that names the entity, an `## Entity purpose` section of prose, an `## Object` table with normative practice levels, a `## YAML Schema Definition` section holding the actual schema, and a `## Sample Object`. The CoreModels `odm` coder reads all five - and this article is the precise account of where each one lands, what travels as annotation rather than structure, which situations produce lossiness records, and why this format is the one place in our lineup where round-trip is deliberately not the goal. - [Microsoft Fabric connector](https://learn.coremodels.io/connectors/fabric.md): Catch the warehouse type change that never pages anyone. - [Microsoft Fabric — Quickstart: From INFORMATION_SCHEMA to a Governed Model: A Microsoft Fabric Walkthrough](https://learn.coremodels.io/connectors/fabric/quickstart.md): Your warehouse already publishes a complete, machine-readable description of itself. Every Fabric Warehouse and SQL analytics endpoint answers `SELECT ... FROM INFORMATION_SCHEMA.COLUMNS` with the tables, the columns, the types, the nullability. Every one of them answers `INFORMATION_SCHEMA.TABLE_CONSTRAINTS` with the keys. That description is the whole input CoreModels needs to build a governed model of the estate and then guard it. - [Microsoft Fabric — API: The Fabric Integration API: Every Route, Every Role, Every Payload](https://learn.coremodels.io/connectors/fabric/api.md): CoreModels exposes vendor integrations over two HTTP surfaces, and the split is not decoration. The interactive surface under `graph/integrations/...` is what a person or a notebook calls with a normal CoreModels login token. The machine-to-machine surface under `v1/...` accepts user API keys and is what a build pipeline calls. Both require authentication; both then enforce a per-project role on top of it. - [Microsoft Fabric — MCP: Governing a Fabric Warehouse from an Agent: the MCP Integration Tools in Practice](https://learn.coremodels.io/connectors/fabric/mcp.md): An agent has a context window, not a file system. That single constraint shapes how AI-driven governance of a Microsoft Fabric warehouse actually works: the model cannot paste a forty-megabyte INFORMATION_SCHEMA extract into a tool call, cannot hold your SQL credentials, and should not be trusted to invent a schema comparison in its head. What it *can* do is call a small set of typed tools that run the real audit engine server-side and hand back a verdict. - [Microsoft Fabric — Automation: Wiring the Drift Gate: Continuous Schema Governance for Fabric and SQL Server](https://learn.coremodels.io/connectors/fabric/automation.md): Schema governance that lives in a document is not governance. It is a document. The version that actually holds is the one a pipeline can fail on - a single number, checked on every change, with a trail behind it that shows whether the estate is getting better or worse. - [Microsoft Fabric — Deep dive: Identity, Types and Honest Loss: What a Fabric Import Actually Writes](https://learn.coremodels.io/connectors/fabric/deep-dive.md): Governance starts with a question that sounds trivial and is not: what *is* a table, once it stops being a table? - [Neo4j connector](https://learn.coremodels.io/connectors/neo4j.md): Write down the graph schema MERGE has been inferring for you. - [Neo4j — Quickstart: Your First Neo4j Schema Audit: A Complete Worked Example](https://learn.coremodels.io/connectors/neo4j/quickstart.md): Take a small graph - three labels, a handful of properties, three relationship types. By the end of this article that graph has a governed model in CoreModels and a recorded baseline audit that tells you, in machine-readable form, exactly what is under governance and where the graph is structurally weak. Total effort: two Cypher statements and two HTTP calls. - [Neo4j — API: The Neo4j Integration API, Route by Route](https://learn.coremodels.io/connectors/neo4j/api.md): Ten routes, two surfaces, one rule about who is allowed to write. That is the core HTTP contract for governing a Neo4j estate with CoreModels, and this article documents it exhaustively - payloads, roles, response shapes, and the failure modes you will actually hit. - [Neo4j — MCP: Four Tools, One Knowledge Graph: Neo4j Governance from the Agent Side](https://learn.coremodels.io/connectors/neo4j/mcp.md): An AI agent connected to CoreModels over MCP sees a small, deliberately shaped set of vendor-integration tools. Three of them cannot write anything at all; the fourth requires Admin membership on the project *and* a different endpoint. That shape is the point of this article: an agent can drive the entire Neo4j governance loop - discover, audit, report, generate the fix - while the only mutating step in the loop stays behind an explicit privilege boundary. - [Neo4j — Automation: Automating Neo4j Drift Control: The Gate, the Trail, the Badge, the Heartbeat](https://learn.coremodels.io/connectors/neo4j/automation.md): Most CI integrations start with an artifact the build already produces. Neo4j does not produce one - there is no compile step for a property graph, no manifest, no migration file that reviewers can read. So the first job in automating Neo4j governance is not writing the gate. It is deciding where `meta_schema.json` comes from in your pipeline. - [Neo4j — Deep dive: Projecting a Property Graph onto a Governed Model: Inside the Neo4j Connector](https://learn.coremodels.io/connectors/neo4j/deep-dive.md): CoreModels has one estate model and every vendor parses into it: **datasets** (table-shaped things) with **fields** and normalized **checks**, plus **lineage** edges and **projections**. A dbt project, a warehouse schema, and a schema registry all land in the same shape. - [ODCS connector](https://learn.coremodels.io/connectors/odcs.md): Make the contract a derived artifact of governed meaning, not a YAML twin of the table. - [ODCS — Quickstart: One Contract, Three Schemas: An ODCS Quickstart](https://learn.coremodels.io/connectors/odcs/quickstart.md): A data contract is a promise with a schema inside it. The ODCS documents your platform team publishes - Open Data Contract Standard, the Bitol project's v3 line - carry tables and columns, but also servers, quality checks, SLAs, ownership: everything the promise needs and most target formats cannot hold. This quickstart takes one real contract through a single CoreModels call three times - to JSON Schema, to SQL, and back to ODCS - and shows you how to read what each conversion kept, what it set aside, and where the set-aside parts went. - [ODCS — API: ODCS Over HTTP: The Transform Routes for Data Contracts](https://learn.coremodels.io/connectors/odcs/api.md): Direction first, because we publish it per format and it decides what you can build. The format key `odcs` appears in both of CoreModels' transform lists - decode (`jsonschema | shex | avro | jsonld | sql | osi | osi-json | owl | linkml | protobuf | odcs | odm`) and encode (`jsonschema | shex | avro | jsonld | sql | osi | osi-json | owl | linkml | protobuf | odcs | synapse`). ODCS round-trips. Its neighbors do not all manage that: `odm` decodes only (ODM entities are authored documentation, and we do not generate prose), and `synapse` encodes only (its output is plain draft-07 JSON Schema - re-import it as `jsonschema`). What `odcs` means here is a Bitol Open Data Contract Standard v3 document - `apiVersion: v3.1.0`, `kind: DataContract` - as YAML or JSON. - [ODCS — MCP: "Turn Our Data Contract Into an Avro Schema": ODCS Through `transform_schema`](https://learn.coremodels.io/connectors/odcs/mcp.md): "The platform team publishes the orders contract - can you make the Avro schema for the streaming team?" That sentence, said to an agent with no tools, produces plausible-looking `.avsc` with invented decisions baked in. Said to an agent connected to CoreModels over MCP, it becomes a call to `transform_schema`: a deterministic engine does the conversion, and the agent gets back the schema, the executed plan, and a machine-readable account of what the trip cost. This article is the complete loop for ODCS - connection, the exact tool contract, two real conversions, and the contract-specific habits that separate a good agent from a confident one. - [ODCS — Automation: The Contract Pipeline Is a Plan File: Automating ODCS With Replay](https://learn.coremodels.io/connectors/odcs/automation.md): Automation is where schema tooling usually stops being honest. A conversion that a human runs once gets its output eyeballed; the same conversion in a nightly job gets trusted. So the two properties that matter most for automating data contracts are not features, they are guarantees: CoreModels transforms are **deterministic** - the same contract in produces byte-identical output every run, which makes diffs a real review surface - and every mapping executes from a **plan** that comes back in the response as JSON you can commit, review, and replay. This article builds an ODCS pipeline on those guarantees: a published subset of an internal contract, regenerated on every change, with drift caught by the engine instead of by a consumer. - [ODCS — Deep dive: Anatomy of the ODCS Coder: What Maps, What Rides, What's Declared](https://learn.coremodels.io/connectors/odcs/deep-dive.md): Where does `servers:` go? - [Open Semantic Interchange connector](https://learn.coremodels.io/connectors/osi.md): Put two revenue numbers next to the same definition. - [Open Semantic Interchange — Quickstart: Ten Minutes to Your First Apache Ossie Transform](https://learn.coremodels.io/connectors/osi/quickstart.md): A semantic model lands in your repository: a YAML file with datasets, fields, a primary key, and a couple of time dimensions. It is the analytics team's description of what the warehouse *means*. Now somebody needs it as a table definition, or as JSON Schema for a validator, or as the JSON serialization dbt ingests. This walkthrough takes you from that file to a converted schema with one HTTP call - and, just as importantly, to a machine-readable ledger of everything the conversion did and did not preserve. - [Open Semantic Interchange — API: The Ossie HTTP Surface: Import, Export, Map, Replay](https://learn.coremodels.io/connectors/osi/api.md): Four verbs cover everything you will do with an Apache Ossie semantic model through the CoreModels API, and picking the right one is mostly a question of *where the schema lives*. Is it a file you have? Import it, or map it statelessly. Is it a governed model in a CoreModels project? Export it. Do you need the same conversion again next month, byte for byte? Replay a stored plan. - [Open Semantic Interchange — MCP: "Convert Our Semantic Model for dbt" - Ossie Through the transform_schema Tool](https://learn.coremodels.io/connectors/osi/mcp.md): Here is the request, as an analyst actually phrases it: *"Take the billing semantic model in our repo, give me the JSON form our dbt project ingests, and tell me what didn't survive the conversion."* - [Open Semantic Interchange — Automation: Ossie Conversions That Belong in CI: Plans as Artifacts, Ledgers as Gates](https://learn.coremodels.io/connectors/osi/automation.md): Run an Ossie conversion twice and diff the two outputs. If the bytes differ, the conversion cannot live in a pipeline - you would be re-reviewing generated files every build. CoreModels (by ARAMAI) gives you the other answer, and it comes from three deliberate design choices: the encoder is hand-written with fixed key order, the mapping plan is returned as an artifact you can store, and replaying a stored plan runs the same validation gate as the call that produced it. - [Open Semantic Interchange — Deep dive: Every Key, Every Record: How the Ossie Coder Maps to the IR](https://learn.coremodels.io/connectors/osi/deep-dive.md): An Apache Ossie semantic model and a typed intermediate representation disagree about the world in one fundamental way: an Ossie field has no type. It is a name, an optional expression, an optional description, and optional dimension metadata. The IR that CoreModels (by ARAMAI) moves schemas through is typed, and so is every neighboring format - SQL, Avro, JSON Schema, LinkML. - [OWL connector](https://learn.coremodels.io/connectors/owl.md): Keep the ontology and the application schema from becoming two portraits of Person. - [OWL — Quickstart: Turtle In, JSON Schema Out: Your First OWL Transform](https://learn.coremodels.io/connectors/owl/quickstart.md): There is a `.ttl` file somewhere in your repository. Someone modeled the domain properly once - classes, subclass axioms, cardinality restrictions, a SKOS scheme for the controlled list - and then everyone else carried on hand-writing JSON Schema and DDL, because nothing bridged the two. - [OWL — API: OWL Over HTTP: Export, Import, Map, Replay](https://learn.coremodels.io/connectors/owl/api.md): Two questions decide whether a format is really supported: can the system read it, and can the system write it? For the `owl` format key in CoreModels the answer is yes to both, and this article is the proof - four routes, real request bodies, real responses, and the direction rules stated without hedging. - [OWL — MCP: Ontology Alignment You Can Delegate: `transform_schema` and OWL Over MCP](https://learn.coremodels.io/connectors/owl/mcp.md): A partner sends you their vocabulary as a Turtle file. You need your catalog expressed in their terms so the two sides can exchange data, and you need a record of which of your concepts had no counterpart on their side. That used to be an afternoon in an ontology editor plus a spreadsheet nobody trusts. - [OWL — Automation: Infer Once, Gate Always, Replay Forever: OWL in a Build Pipeline](https://learn.coremodels.io/connectors/owl/automation.md): Commit an exported ontology, re-run the export tomorrow, and look at the diff. If it shows reordered prefixes and reshuffled blocks, the export is not a build artifact - it is a rumor, and no reviewer will read it twice. If it shows exactly the statements your model changed, you have something CI can gate on. - [OWL — Deep dive: What the OWL Coder Knows: The IR Map, the Extras, the Ledger](https://learn.coremodels.io/connectors/owl/deep-dive.md): Ask a schema converter what it does with `owl:Restriction` and you learn most of what you need to know about it. Many treat OWL as a serialization problem - emit classes and properties, drop the axioms - so "required" and "single-valued" evaporate at the border. CoreModels treats it as a semantics problem: required and cardinality ride as `owl:Restriction` subclass axioms, OWL's own carrier, in **both** directions. That decision propagates through everything below. - [Protocol Buffers connector](https://learn.coremodels.io/connectors/protobuf.md): Carry reserved field numbers with the meaning, not only with the .proto file. - [Protocol Buffers — Quickstart: Ten Minutes With proto3: Your First CoreModels Transform](https://learn.coremodels.io/connectors/protobuf/quickstart.md): You have a `.proto` file. It is the truth for a service, and now somebody outside that service needs the same shape - as JSON Schema for a validation step, as a data contract for a review, as a table for analytics. The usual answer is to retype it. This article replaces the retyping with one HTTP call, and then spends most of its length on the part that matters more: reading what the conversion cost you. - [Protocol Buffers — API: The Transform API for proto3: Four Routes, Both Directions](https://learn.coremodels.io/connectors/protobuf/api.md): Most schema converters are one-way. You can get *out* of the format or *into* it, rarely both, and the documentation usually leaves you to discover which by trying. So let us answer the direction question for Protocol Buffers first and in one sentence: **proto3 decodes and encodes.** The key `protobuf` (alias `proto`) appears in both the decode list and the encode list, so it is valid as a source and as a target on every route below. - [Protocol Buffers — MCP: Handing proto3 to an Agent: transform_schema over MCP](https://learn.coremodels.io/connectors/protobuf/mcp.md): "Take our telemetry `.proto` and give me the Postgres table for it - and tell me what we lose." - [Protocol Buffers — Automation: Deterministic proto3 Pipelines: The Plan Is the Artifact](https://learn.coremodels.io/connectors/protobuf/automation.md): A conversion you run once is a favor. A conversion you run on every commit is infrastructure, and infrastructure needs two properties that one-off tooling never has to prove: the second run must produce the same bytes as the first, and when the input changes in a way the conversion cannot honor, the pipeline must stop rather than quietly produce something different. - [Protocol Buffers — Deep dive: Inside the proto3 Coder: The IR Mapping, the Extras, and Every Lossiness Record](https://learn.coremodels.io/connectors/protobuf/deep-dive.md): Proto3 has fifteen scalar types. A neutral intermediate representation that also hosts JSON Schema, SQL, Avro, LinkML, OWL, and half a dozen other formats cannot afford fifteen distinct numeric primitives, and any tool that claims otherwise is hiding something. The arithmetic settles the question up front: some proto3 facts map structurally, some ride alongside as preserved detail, and some are declared as loss. This is the complete account, in that order. - [REDCap connector](https://learn.coremodels.io/connectors/redcap.md): Diff the codebook so a study change is a review, not a memory. - [REDCap — Quickstart: REDCap to CoreModels: Zero to First Audit with One CSV](https://learn.coremodels.io/connectors/redcap/quickstart.md): Most schema estates take real work to extract: information-schema queries, catalog exports, manifest builds. A REDCap project is the pleasant exception - the entire schema estate is one stable file, the data dictionary CSV, and you almost certainly already know how to download it. That makes the distance from "nothing governed" to "first drift audit" unusually short. This tutorial walks the whole distance: export the dictionary, import it into a governed CoreModels project, run your first audit, and read what comes back. No REDCap credentials are ever shared with CoreModels at any point. - [REDCap — API: Every REDCap Route in CoreModels: the Full HTTP Reference](https://learn.coremodels.io/connectors/redcap/api.md): The CoreModels REDCap connector declares all three capabilities - **Import, Audit, Generate** - which means every integration verb on our HTTP surface is live for it. This article is the complete route-by-route reference: what each endpoint takes, what it returns, which role it requires, and which of the two HTTP surfaces it lives on. If you're writing a script, a service, or an internal tool against the REDCap integration, this is the page to keep open. - [REDCap — MCP: Give Your Agent the Dictionary: REDCap Governance over MCP](https://learn.coremodels.io/connectors/redcap/mcp.md): When the caller is an AI agent rather than a shell script, the interesting question stops being "what's the route?" and becomes "what is the agent allowed to know, and what is it allowed to do?" CoreModels answers that with an MCP server whose vendor-integration tools mirror our HTTP surface exactly - same audit engine, same additive import, same read-only posture - but packaged as typed tools an agent can discover, reason about, and chain. This article walks the four tools that matter for REDCap, with the real arguments, and shows the pattern we recommend for agents that govern clinical research schemas: discover, audit, and only then (with a human's admin endpoint) import. - [REDCap — Automation: The Dictionary Changed on a Tuesday: CI Drift Gates for REDCap](https://learn.coremodels.io/connectors/redcap/automation.md): REDCap's greatest operational strength is also its governance problem: a data manager with the right permissions can change a production instrument in minutes. A validation type quietly switches from `integer` to plain text, a choice list gains a code, a required flag disappears - and every downstream extract, harmonization script, and statistical pipeline inherits the change without a review. The fix isn't to slow REDCap down. It's to make the data dictionary a versioned, gated artifact like any other piece of production configuration. This article builds that gate with CoreModels: a CI audit that fails on drift, a status badge, a rolling history, one-call re-audits, and the scheduled heartbeat that catches drift nobody pushed. - [REDCap — Deep dive: Inside the REDCap Connector: How a Data Dictionary Becomes a Governed Graph](https://learn.coremodels.io/connectors/redcap/deep-dive.md): To trust a governance tool you should be able to predict what it does with your data - row by row, column by column. This deep dive opens the hood on the CoreModels REDCap connector: how the data dictionary CSV is parsed, how each REDCap concept lands in the governed graph, which facts ride vendor metadata rather than governed meaning, what triggers each audit rule, and precisely where the mapping is lossy. - [Salesforce connector](https://learn.coremodels.io/connectors/salesforce.md): Govern the org whose schema can change in forty seconds from Setup. - [Salesforce — Quickstart: Governing a Salesforce Org in an Afternoon: From describe.json to Your First Drift Audit](https://learn.coremodels.io/connectors/salesforce/quickstart.md): Somewhere in your org, last quarter, somebody edited a picklist. Nobody remembers who, the field history doesn't say why, and the report that broke three weeks later never mentioned it at all. Salesforce makes schema change wonderfully easy - and makes *remembering what the schema is supposed to mean* entirely your problem. - [Salesforce — API: Seven Verbs and Two Surfaces: The Complete Salesforce Integration API](https://learn.coremodels.io/connectors/salesforce/api.md): Before writing a single call, it pays to see the whole map. The Salesforce integration in CoreModels (vendor key `salesforce`) is small enough to hold in your head: the seven core per-vendor verbs on the interactive HTTP surface, two on the machine-to-machine surface, one artifact, and two roles. The connector declares all three capabilities - **Import, Audit, Generate** - so every verb below is live for Salesforce; nothing in this reference is aspirational. - [Salesforce — MCP: "Did the Org Drift?" - Running Salesforce Governance Through an AI Agent](https://learn.coremodels.io/connectors/salesforce/mcp.md): Watch an agent handle a governance request end to end. A data engineer types: *"Here's this week's describe export - check whether the Salesforce org still matches the governed model, and summarize anything that changed."* The agent calls one tool, reads back structured findings plus a ready-made markdown report, and answers with specifics: which object, which field, which severity. No dashboard visit, no memorized route, no guessing. - [Salesforce — Automation: The Build Goes Red Before the Org Goes Wrong: A Salesforce Drift Gate in CI](https://learn.coremodels.io/connectors/salesforce/automation.md): Schema governance that lives in a wiki dies in a wiki. The only governance that survives contact with a delivery team is the kind wired into the pipeline - a check that runs on every change, fails loudly when meaning breaks, and costs nothing to keep passing. This article wires that check up for Salesforce with CoreModels: a CI job that audits a fresh describe export against the governed model, a one-line pass/fail contract, a status badge, a rolling evidence trail, and the two mechanisms that cover what CI cannot see. - [Salesforce — Deep dive: Anatomy of a Governed Org: How Salesforce Metadata Maps into the CoreModels Graph](https://learn.coremodels.io/connectors/salesforce/deep-dive.md): A drift audit is only as trustworthy as the mapping underneath it. If an import flattens your org into undifferentiated strings, the audit can only ever tell you undifferentiated things. So this article opens the hood on the CoreModels Salesforce connector: what each describe property becomes in the governed graph, which platform semantics survive the crossing, which are approximated - and how every approximation is declared rather than hidden. - [ShEx connector](https://learn.coremodels.io/connectors/shex.md): Keep the four assertions in every shape constraint from dying in a hand copy. - [ShEx — Quickstart: ShEx in Ten Lines: Your First CoreModels Transform](https://learn.coremodels.io/connectors/shex/quickstart.md): The shortest useful thing you can do with CoreModels and ShEx takes one HTTP call, writes nothing to your project, and hands back three things: the converted schema, a replayable plan, and an honest ledger of everything the conversion could not carry across. This walkthrough runs that call twice - once into JSON Schema, once into Postgres DDL - and shows you how to read all three parts of the answer. - [ShEx — API: The ShEx HTTP Surface: Import, Export, Map, and Replay](https://learn.coremodels.io/connectors/shex/api.md): Five HTTP endpoints carry ShEx through CoreModels, and they are the same five that carry every other schema format we support - nothing about ShEx is a special case on the wire. What changes is one string: the format key `shex`. This article is the contract reference for that surface: exact routes, request and response bodies, roles, options, and the honest statement of what each direction can and cannot do. - [ShEx — MCP: Agent-Driven ShEx Conversion with the transform_schema MCP Tool](https://learn.coremodels.io/connectors/shex/mcp.md): "The ontology team published ShEx shapes for the product catalog. Give me the JSON Schema our validation service needs, and tell me exactly what didn't survive the trip." - [ShEx — Automation: ShEx in the Pipeline: Deterministic Transforms, Replayable Plans, Batch Conversion](https://learn.coremodels.io/connectors/shex/automation.md): A schema conversion you cannot reproduce is a liability with a timestamp on it. If your ShEx shapes are converted to JSON Schema by hand, or by a script whose behavior depends on who runs it and when, then the day the outputs disagree you have no way to say *which* conversion was right. We built the CoreModels transform surface so that conversion can live in a pipeline like any other build step: deterministic outputs, a reviewable plan artifact you can commit next to your shapes, and a change report you can gate a build on. This article shows the working patterns for ShEx (format key `shex`). - [ShEx — Deep dive: Inside the ShEx Coder: The IR Map, the Extras Ledger, and What Round-Trips](https://learn.coremodels.io/connectors/shex/deep-dive.md): Most schema formats name things locally: a SQL column is `full_name`, an Avro field is `full_name`, and any connection to the wider world has to be bolted on with annotations. ShEx arrives different. Its predicates and shape names are IRIs - `schema:name` *is* `https://schema.org/name` - which means a ShEx document carries its cross-standard identity in its bones. Our ShEx coder in CoreModels, by ARAMAI, is built around that fact, and this article is the full technical account of it: exactly what maps into our intermediate representation (IR), what rides in the preserved-extras ledger, every lossiness record the coder can emit, and precisely what survives a round trip. - [Snowflake connector](https://learn.coremodels.io/connectors/snowflake.md): Your Snowflake schema is not a contract. Generate one from the governed model. - [Snowflake — Quickstart: From Snowsight to First Audit: Governing a Snowflake Schema with CoreModels](https://learn.coremodels.io/connectors/snowflake/quickstart.md): You have a Snowflake schema and about thirty minutes. By the end of this tutorial you will have a governed model of that schema in CoreModels - tables as Types, columns as Elements with their native types preserved, declared keys as checks and references, object dependencies as lineage - and you will have run your first drift audit against it and read the result. - [Snowflake — API: Every Endpoint: The CoreModels HTTP Surface for Snowflake](https://learn.coremodels.io/connectors/snowflake/api.md): This is the reference walk through everything you can do with the Snowflake integration over plain HTTP - every route, its role requirement, its request body, and what comes back. The Snowflake connector declares all three capabilities - Import, Audit, and Generate - so every verb below is live for it. - [Snowflake — MCP: An Agent Runs Your Snowflake Governance: The MCP Workflow](https://learn.coremodels.io/connectors/snowflake/mcp.md): "Is our Snowflake estate still in sync with the governed model? If not, show me exactly what drifted, and draft the DDL to fix it." That is a sentence you can now say to an AI agent, and every step it takes to answer - discovery, audit, report, generation - runs through the CoreModels MCP server as first-class tool calls. This article walks the whole flow: how an agent connects, which integration tools it gets, the exact arguments each takes, and how large metadata extracts travel via the `artifactUrls` path. - [Snowflake — Automation: Wiring a Snowflake Drift Gate: CI, the Badge, and the Rolling Trail](https://learn.coremodels.io/connectors/snowflake/automation.md): Schema drift has two directions, and most teams only guard one. The warehouse can move away from the agreed model - a column retyped in a Friday migration, a table quietly dropped. But the model can also move away from the warehouse - someone tightens an allowed-value list in governance and nobody re-checks the estate. This article wires up automation for both directions against a Snowflake estate governed in CoreModels: a CI gate on the machine-to-machine API, a status badge, the rolling audit history, one-call re-audits, and the scheduled heartbeat that keeps the loop honest between changes. - [Snowflake — Deep dive: Anatomy of an Import: How a Snowflake Estate Becomes a Governed Graph](https://learn.coremodels.io/connectors/snowflake/deep-dive.md): Take one column - `ORDER_TOTAL NUMBER(38,2)` in `ANALYTICS.PUBLIC.ORDERS` - and follow it from an `INFORMATION_SCHEMA` row to a governed graph node. That single trace touches everything interesting about the Snowflake connector in CoreModels: how identity is minted, how native types map (and where they approximate), what rides the vendor-metadata mixin, how keys and dependencies become checks and lineage, which audit rules are Snowflake's own, and where the connector honestly declines to guess. This article is that trace, written for people who want to know exactly what the import does before pointing it at production metadata. - [SQL connector](https://learn.coremodels.io/connectors/sql.md): Give the column nobody can explain a rule the database can actually carry. - [SQL — Quickstart: Your First DDL Transform Should Write Nothing: a SQL Quickstart](https://learn.coremodels.io/connectors/sql/quickstart.md): The most accurate description of your data is almost certainly a `CREATE TABLE` statement sitting in a migrations folder. It is reviewed, versioned, and executable - and it is legible to exactly one kind of consumer. Getting it out of that folder and into a data contract, a JSON Schema, or another vendor's dialect is usually where the accuracy stops. - [SQL — API: Four Routes, One Envelope: SQL DDL on the CoreModels Transform API](https://learn.coremodels.io/connectors/sql/api.md): | Route | Role | What it touches | |---|---|---| | `POST /graph/transform/schema/import/{projectId}` | Admin | writes the DDL's model into the project | | `POST /graph/transform/schema/export/{projectId}` | Viewer | reads the project, returns `CREATE TABLE` text | | `POST /graph/transform/schema/map/{projectId}` | Viewer (`ai`: Editor) | nothing - stateless, source in, target out | | `POST /graph/transform/plan/execute/{projectId}` | Viewer | nothing - replays a stored plan | - [SQL — MCP: Teaching an Agent to Read DDL: the transform_schema MCP Tool](https://learn.coremodels.io/connectors/sql/mcp.md): A partner attaches `subscriber_dump.sql` to a ticket. Backticks, an `ENUM`, a `DATETIME`, and one column named by a DBA who shouts. Your canonical model is a JSON Schema in a repo. Somebody has to reconcile the two, and historically that somebody spent an afternoon in a diff viewer. - [SQL — Automation: Commit the Plan, Not the Guess: SQL Conversion Pipelines That Repeat](https://learn.coremodels.io/connectors/sql/automation.md): Database teams solved schema-as-code years ago. DDL lives in git, migrations get reviewed, nobody types `ALTER TABLE` into production by hand. Then the schema has to leave the database - become a JSON Schema for the API team, a contract for the platform team, DDL for a second warehouse - and the discipline evaporates into a conversion script that one person ran once on a laptop. - [SQL — Deep dive: No Annotation Slot: How the SQL DDL Coder Carries Meaning, and Where It Stops](https://learn.coremodels.io/connectors/sql/deep-dive.md): Every other format on the CoreModels transform surface has somewhere to put meaning. JSON Schema has `x-` keywords. Avro has custom attributes. LinkML has `slot_uri` and `meaning`. OWL *is* meaning. SQL DDL has a name, a type, and a handful of constraints - and no annotation slot at all. --- # Grounding an assistant Source: https://learn.coremodels.io/agents/grounding An assistant writing SQL against your warehouse has two ways to decide what `status` means. It can infer from the column name and the rows it can see, or it can look the definition up. Only one of those is repeatable. ## The failure this fixes Ask a coding assistant to write a query filtering on order status and it will produce something plausible: `WHERE status IN ('shipped', 'completed')`. Plausible is the problem. If your estate also uses `return_pending`, the query is quietly wrong, it passes review because it reads correctly, and nothing in your pipeline objects. The model is not being careless. Nothing in the schema it was given carries the allowed values, so it filled the gap with the most likely guess. Give it somewhere to look and the guess disappears. ## The pattern 1. **Model the meaning once.** Bring the schema your stack already produces, then fix the definitions in one place — the allowed values, the owner, what the field is actually for. 2. **Attach the MCP server** so the assistant can read that model at the moment it writes code. See [MCP server](/agents/mcp). 3. **Tell the assistant to consult it before asserting.** The instruction that does the work is plain: *look up before you make up*. Put it in your project rules, your system prompt, or your agent's standing instructions. 4. **Make lookups cheap.** Point the assistant at the specific model for the repository it is working in, not at everything. A narrow, correct source beats a broad one it will not read. ## A rule you can paste ```text Before you assert what a field means, allows, or is used for, look it up in CoreModels via the MCP server. If the model has no definition for it, say so — do not infer one from the column name. ``` That last clause matters more than the first. An assistant that reports a gap gives you something to fix; one that infers gives you a bug shaped like an answer. ## Where to go next Every connector documents its own MCP path — how to expose that estate's model to an agent. The dbt one is the fullest worked example: [dbt → MCP](/connectors/dbt/mcp). --- # MCP server Source: https://learn.coremodels.io/agents/mcp ## The endpoint ``` https://go.coremodels.io/mcp ``` One endpoint serves both reads and writes. Which operations a key may perform is decided by the key, not by the URL. ## Authentication Authenticate with a bearer token — an API key you generate in CoreModels under **account settings → API keys**. ```json { "mcpServers": { "coremodels": { "url": "https://go.coremodels.io/mcp", "headers": { "Authorization": "Bearer YOUR_API_KEY" } } } } ``` That block goes in your client's MCP configuration — `mcp.json` for Cursor, the MCP settings file for Claude Desktop, or the equivalent for whichever client you use. > **Note.** The server answers an unauthenticated request with `401 Unauthorized` and a > `WWW-Authenticate: Bearer` challenge, including on the initial handshake. Configure a key before > your first call, even if you only intend to read. ## Limits Each agent gets up to **15 concurrent in-flight requests**. Above that it receives a rate-limit response and should retry. Enterprise plans set throughput in contract. Plan limits on model size and project count are on [Plans and limits](/account/plans). ## What to do with it An assistant with this server attached can search your models, read a type and its elements, and — with a key that permits writes — create and update them. The point is not the tool list. The point is that the assistant stops guessing what your columns mean, because it can look the answer up. Start with [Grounding an assistant](/agents/grounding), then use the [tool reference](/agents/tools) for signatures. --- # Tool reference Source: https://learn.coremodels.io/agents/tools The tool list is produced by querying `https://go.coremodels.io/mcp` at build time. That call needs an API key, and none was available when this build ran, so the list is empty here. Connect your own client and ask it to list the tools — see [MCP server setup](https://learn.coremodels.io/agents/mcp). --- # Connectors Source: https://learn.coremodels.io/connectors ## Format - [Apache Avro](https://learn.coremodels.io/connectors/avro) — One governed reading of the schema that currently lives in four places. - [JSON Schema](https://learn.coremodels.io/connectors/jsonschema) — Stop asking a validator four questions it was only built to answer one of. - [JSON-LD](https://learn.coremodels.io/connectors/jsonld) — Make the vocabulary you already trust executable downstream. - [LinkML](https://learn.coremodels.io/connectors/linkml) — Let the pipeline read the model that is already right. - [MACH ODM](https://learn.coremodels.io/connectors/odm) — Stop shipping a recollection of the standard as if it were the standard. - [ODCS](https://learn.coremodels.io/connectors/odcs) — Make the contract a derived artifact of governed meaning, not a YAML twin of the table. - [OWL](https://learn.coremodels.io/connectors/owl) — Keep the ontology and the application schema from becoming two portraits of Person. - [Protocol Buffers](https://learn.coremodels.io/connectors/protobuf) — Carry reserved field numbers with the meaning, not only with the .proto file. - [ShEx](https://learn.coremodels.io/connectors/shex) — Keep the four assertions in every shape constraint from dying in a hand copy. - [SQL](https://learn.coremodels.io/connectors/sql) — Give the column nobody can explain a rule the database can actually carry. ## Platform - [Airbyte](https://learn.coremodels.io/connectors/airbyte) — Govern the catalog every sync trusts - before a silent type change lands in the warehouse. - [Apache Airflow](https://learn.coremodels.io/connectors/airflow) — Govern the scheduler that already knows your freshness, ownership, and lineage. - [AWS Glue](https://learn.coremodels.io/connectors/glue) — Stop treating a crawler snapshot as the meaning of your lake. - [Azure Synapse](https://learn.coremodels.io/connectors/synapse) — Record the constraints that die on the way to a registered schema. - [cBioPortal](https://learn.coremodels.io/connectors/cbioportal) — Govern the four comment rows that are the schema of your study. - [Confluent](https://learn.coremodels.io/connectors/confluent) — Put meaning under every Kafka subject the Schema Registry already inventories. - [Databricks](https://learn.coremodels.io/connectors/databricks) — Give Unity Catalog something to check thousands of columns against. - [dbt](https://learn.coremodels.io/connectors/dbt) — Meaning has no home in dbt. Bring a manifest; we turn `accepted_values` into a vocabulary you can govern and your agent can read. - [Google BigQuery](https://learn.coremodels.io/connectors/bigquery) — Turn INFORMATION_SCHEMA from an inventory into a contract you can audit. - [Microsoft Fabric](https://learn.coremodels.io/connectors/fabric) — Catch the warehouse type change that never pages anyone. - [Neo4j](https://learn.coremodels.io/connectors/neo4j) — Write down the graph schema MERGE has been inferring for you. - [Open Semantic Interchange](https://learn.coremodels.io/connectors/osi) — Put two revenue numbers next to the same definition. - [REDCap](https://learn.coremodels.io/connectors/redcap) — Diff the codebook so a study change is a review, not a memory. - [Salesforce](https://learn.coremodels.io/connectors/salesforce) — Govern the org whose schema can change in forty seconds from Setup. - [Snowflake](https://learn.coremodels.io/connectors/snowflake) — Your Snowflake schema is not a contract. Generate one from the governed model. --- # Changelog Source: https://learn.coremodels.io/changelog ## v4.0.0 — 3 September 2026 - **dbt recipes (beta).** Seven guided runs over a dbt project: governed vocabulary, publication contract, contract of record and others. See [the dbt connector](/connectors/dbt). - **Avro and SQL import and export.** Both formats now round-trip through a Core Model. - **MCP fixes.** Corrections to the [MCP server](/agents/mcp) surface. ## Earlier This changelog starts with v4.0.0 — the first release after the documentation rebuild. Earlier release notes were not kept in a form worth republishing, and inventing them would be worse than starting here. --- # Roles Source: https://learn.coremodels.io/account/roles CoreModels provides the following roles, listed in order of authority. Each role inherits the capabilities of the roles preceding it: #### Viewer Viewers are read-only members of a project. They can: * View all models, schemas, and content. * Export data and schemas. * View all comments. *** #### Collaborator Collaborators can view and contribute to the project's content. They have all Viewer permissions, and can also: * Update and edit the project's content. * Add and reply to comments. *** #### Editor Editors can update the core structure of the schema. They have all Collaborator permissions, and can also: * Edit and update the entire schema, including **Relations** and **Mixins**. *** #### Admin & Owner Admins have full control over the project and its members. They have all Editor permissions, and can also: * Import and export data. * Manage project members and their roles. * Connect the project to external integrations. * Perform backups and restore the project. * Permanently delete the project. --- # Neo AI Agent Source: https://learn.coremodels.io/agents/neo ## Capabilities ### Reading Data Neo can retrieve and explore your CoreModels project's schemas, and entities through conversational queries. ### Creating Data Neo can create new schemas and entities in CoreModels based on your instructions. ### File Attachment Neo supports file attachments in conversations, allowing you to upload documents for context or reference during your interactions. *** ## Example Prompts ### Creating Data This example shows Neo creating a Person schema in CoreModels from an uploaded (attached) JSON-LD file. *** 📎 **File attached:** person schema.txt ```json { "@context": "https://schema.org", "@type": "Person", "@id": "https://example.com/people/jane-doe", "name": "Jane Doe", "givenName": "Jane", "familyName": "Doe", "alternateName": "J. Doe", "birthDate": "2005-08-17", "gender": "Female", "email": "mailto:jane.doe@example.com", "telephone": "+1-555-123-4567", "url": "https://example.com/jane-doe", "image": "https://example.com/images/jane.jpg", "sameAs": [ "https://www.wikidata.org/wiki/Q123456", "https://www.instagram.com/janedoe" ], "address": { "@type": "PostalAddress", "addressLocality": "New York", "addressRegion": "NY", "addressCountry": "US" } } ``` > **User** > > create the schema found in the attached file inside coremodels. > **Neo** > > I'll help you create the Person schema from the attached file in CoreModels. Looking at the JSON-LD schema, I can see we have a Person type with various properties, and a nested PostalAddress type for the address. > > Let me create the schema structure with the appropriate types and elements: `🔧 Tool called: core_models_bulk_create` > **Neo** > > Perfect! I've successfully created the Person schema from your attached file in CoreModels. Here's what I created: > > Schema Structure Created > > Types > > * **Person** - The main type representing a person > * **PostalAddress** - Type for address information > > Person Properties (Elements) > > * **name** - Full name > * **givenName** - First name > * **familyName** - Last name > * **alternateName** - Alternative name (like "J. Doe") > * **birthDate** - Date of birth > * **gender** - Gender > * **email** - Email address > * **telephone** - Phone number > * **url** - Personal website URL > * **image** - Profile image URL > * **sameAs** - Links to other profiles (social media, etc.) > * **address** - Address information (links to PostalAddress type) > > PostalAddress Properties (Elements) > > * **addressLocality** - City (like "New York") > * **addressRegion** - State/Region (like "NY") > * **addressCountry** - Country (like "US") > > Relationships Created > > * All Person properties are linked to the Person type using "Domain Includes" relations > * All PostalAddress properties are linked to the PostalAddress type using "Domain Includes" relations > * The address property has a "Range Includes" relation to the PostalAddress type, indicating it can contain PostalAddress objects > > The schema structure now matches your JSON-LD Person schema and is ready to use in CoreModels! You can extend it further by adding more properties, creating exemplars with sample data, or adding additional constraints using mixins. Here is the result in CoreModels ![](/docs-assets/FEmHBRV5A0PfCGCW4mDy.png) --- # Integration APIs Source: https://learn.coremodels.io/api ### API Key Generation From the top right corner, tap your profile picture, then click on 'User Settings' ![](/docs-assets/EDaigUpIv7ng3eACBISt.png) Th User Settings modal will show up, click on 'API Management', then click on the Generate button, to generate a new Key. Copy and store the Key as it will not be displayed again. Notes: * As of v3.1 users can only have one active API Key, and if you need to re-generate a new key, it will revoke the old one. * API Key life time is 3 months after that it'll be automatically revoked. * All operations will be logged in change log using the user account the API key belongs to. * The API Key has the same owner user permissions. ### Integration APIs To view the available APIs, open the channels in the features tray, and click on 'RESTful' to open the tab. ![Accessing the RESTful Tab](/docs-assets/5e28mCud0lj7wm8muQe3.png) *Accessing the RESTful Tab* The Swagger interface shows up, you view the available APIs and the DTO models used, you can also try the APIs. To test the APIs, first click on the 'Authorize' green button, then enter 'Bearer' \[space] and then your token. ### Guide CoreModels is a dynamic graph based solution where users can create their own spaces, mixins, and relation groups, and most APIs are referencing those by their Ids. (Both the required DTOs when creating nodes or relations, and the API results when requesting to fetch nodes are referring to the spaces, mixins, and relation groups by their Id) Therefor the first step for developers is to call and store this information first, as those Ids will be used in most other APIs: 1. `relationsGroupsInfo` will return the relation groups. 2. `mixinsInfo` will return the mixins. 3. `fetchNodes` with `"nodeType":"Space"` will return the spaces. #### Fetching The model Developers can either use the `fetchNode` API to return a single node by its Id, or the general `fetchNodes` API to search for nodes based on a criteria. Both APIs offer two parameters `addRelations` and `addMixins` that will include more information about the node. `fetchNodes` can be used to: * Return nodes of a specific kind (types, elements, or taxonomies etc...) with token based pagination. * Return nodes based on an expression that will filter the nodes based on the label. * Return nodes by their Ids. #### Modifying the model To modify the nodes: use the `node` endpoints: * Post is for creating new nodes. * Put is to update a node (by its Id) * Delete is for removal To modify relations: use the `relation` endpoints: * Post is for creating new relations * Delete is for removal To modify mixins values for a specific node use the `mixins` (POST) endpoint. Please refere to the complete [Reference ](/api/reference)for more details on each API. --- # Reference Source: https://learn.coremodels.io/api/reference ## 1. Fetch Operations ### 1.1 Fetch Node Fetch a single node’s info using its ID **GET** `/v1/{projectId}/fetchNode` [IntegrationAPIs\_swagger.json](/openapi.json) * Path parameters * projectId: * CoreModels projectId can be copied from the settings page * or it can be inferred from the URL: ![](/docs-assets/0SMc92TW0FVFSiTgBP2p.png) * Query parameters * NodeId: determines the node Id you want to get information about (required parameter). * AddRelations: will include node relations in the result (optional parameter, by default is false). * AddMixins: will include node mixins ( for existing values only ) in the result (optional parameter, by default is false). * Response * success: determine if operation succeeded or not. * error: if any error occurred you'll get an error object, with * isFata: to determine if this is a fatal error in the system, or it's a logical error. * message: if error is fatal: you'll get error message to contact the admin with specific error code for further tracking, else if the error is not fatal, then the error message will determine what is the logical error in the request. * data: contains info about the fetched node. * Id: node Id in CoreModels. * label: node label in CoreModels. * NodeType: can be one of: Element, Type, Taxonomy, Exemplar, Component, Space, Tag, Mixin (case sensitive value). * spaceIds: in which space(s) this node is located. * relations: list all node relations, will be empty if AddRelations is false in the request, or node has no relations. * mixins: list all node mixins, will be empty is AddMixins is false in the request, or node has no mixins. ### 1.2 Fetch Nodes Paginated fetch and search operation for any node in CoreModels. **GET** `/v1/{projectId}/fetchNodes` [IntegrationAPIs\_swagger.json](/openapi.json) * Path parameters: * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Query parameters: * NodeType: see [#fetch-nodes](#fetch-nodes "mention"). * NodeIds: node Ids you want to search for. * Expression: search by part of the node name. Note: you've to provide at least one of NodeType, NodeIds, and Expression, but also you can use any combination of them to perform more complex filter for your search. * SpaceIds: in which space(s) you want to search, if empty it means the main space. * AddRelations: add node relations to the result see[#fetch-nodes](#fetch-nodes "mention"). * AddMixins: add node mixins to the result see [#fetch-nodes](#fetch-nodes "mention"). * SortAttr: Label, CreationTime, or UpdateTime (sort the results base on the selected criteria). * sortDesc: true for sort results descending (default false if not provided). * pageSize: the page size of the fetch paginated search operation: must be in range (1,250) default 50 if not provided. * pagingToken: the pagination token. see the response parameters section below. * indeludeRelationsFromAllSpaces: default to false, if send true: the relations of all spaces will be returned with node result. * Response: * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data * nodes: the result nodes. * lastPage: true if this is the last page of the result. * pagingToken: if lastPage is false, then use the pagingToken value when you send the next request to get the next page. so you'll send the same request again with same parameters but set pagingToken to the new value to get the next page.\ known limitation: we don't return page count of the fetch result. ## 2. Nodes Operations ### 2.1 Create Node Creates Type, Element, Taxonomy, Exemplar, Component (view), Space, Tag, or Mixin in CoreModels. **POST** `/v1/{projectId}/node` [IntegrationAPIs\_swagger.json](/openapi.json) * path parameters: * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Request Body: * Id: NodeId you want to create, if left empty it'll be generated for you, and returned in the result. We prefer GUID shaped values if you want to provide it. * label: node label. * nodeType: for accepted node types see [#fetch-nodes](#fetch-nodes "mention") * checkBeforeCreate: if true, we'll check for the label if used by any other node, and return the check result in checkResult object of the result.\ Note: even if the node label already exists the new node will be created. * spaceIds: the space(s) where you want to locate the newly created node, if left empty it'll be created in the main spaec. * Response: * Id: the created node Id. * checkResult: if checkBeforeCreate set to true in the request, this will hold list of checkResult objects. * checkResult content: result of check as string. * level: warning always. ### 2.2 Update Node Update a node’s label and space (optionally) using its ID. **PUT** `/v1/{projectId}/node` [IntegrationAPIs\_swagger.json](/openapi.json) * Path parameters: * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Request body * Id: Id of node you want to edit. * label: new label value. * spaceIds: (optional) set the new spaces for the node, if left empty: the spaces will be kept as is. * Resposnse * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data: always empty for now. ### 2.3 Remove Node Remove a node using its ID. **DELETE** `/v1/{projectId}/node/{id}` [IntegrationAPIs\_swagger.json](/openapi.json) * Path parameters * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Id: node Id to remove (soft delete). * Reponse * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data: always empty for now. ## 3. Relations ### 3.1 Relation Groups info In CoreModels there are default relation groups, and user also can add custom relation groups. In order to create any relation between two nodes in CoreModels you need the relation Group Id to determine the type of this relation. We suggest you to retrieve the Relation Groups info using this API, store their info for later usage with relation operations. **GET** `/v1/{projectId}/relationGroupsInfo` [IntegrationAPIs\_swagger.json](/openapi.json) * Path parameters * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Reposnse * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data: array of relation info, each object consists of * label: relation group name (label). * relationGroupId: the Id of the relation Group, this is important because we work with Ids when we want to create any relation. * from: the accepted types for the ***From*** node part of the relation. ( can by one or more of NodeTypes). * to: the accepted types for the ***To*** part of the relation. (can be one or more of NodeTypes). * secondLevelFrom & secondLevelTo: not used for now. ### 3.2 Create Relation to create a new relations between two nodes in CoreModels, please provide: *from* node Id, *to* node Id, and relation Group Id (explained in the previous section). **POST** `/v1/{projectId}/relation` [IntegrationAPIs\_swagger.json](/openapi.json) * Path parameters * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Request body * Id: required Id of the newly create relation we suggest to use GUID shaped values for all Ids, if this value not provided we'll create the new Id and return it in the result. * fromNodeId: node Id where relations starts. * toNodeId: node Id where relation ends. * relationGroupId: the Id of relation group (which determine the type of the relation). * value: not used for now. * Resposen * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * date: * Id: the Id of the created relation. Note: there will be a validation step before creating the required relation, if both (from, to) nodes fulfill the relation group, the relation will be created else an error is returned with the appropriate message. ### 3.3 Remove Relation Remove a relation using its info. **DELETE** `/v1/{projectId}/relation` [IntegrationAPIs\_swagger.json](/openapi.json) * Path parameters * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Request body * same value when relation created, see [#create-relation](#create-relation "mention") * Response * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data: empty for now. ## 4. Mixins ### 4.1 Get Mixins Info In CoreModels there are default Mixins, and user also can add custom mixins. In order to add/update/remove a mixin for any node in CoreModels you need the Mixin Id, and It's columns Ids. We suggest you to retrieve the Mixins info using this API, store their info for later usage with mixins operations. **GET** `/v1/{projectId}/mixinsInfo` [IntegrationAPIs\_swagger.json](/openapi.json) * Path parameters * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Response * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data: list of mixin, each mixin consists of * Id: the mixin Id. * label: mixin name (label) * type: mixin type: multi level mixin, attribute mixin, or normal mixin. * columns: list of mixin column; to represents the columns in this mixin, each column has this properties: * Id: column Id. * Label: column name (label). ### 4.2 Create Mixin This sets the mixin value for a specific node. You need to determine the Mixin Id, and then what are the columns you need to set values (using column Id, and the needed new value for the column). **POST** `/v1/{projectId}/mixinValue` [IntegrationAPIs\_swagger.json](/openapi.json) * Path parameters * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Request body * target Node Id: for what node you want to set the mixin value. * mixinId: which mixin you want to change. * mixinColumns: determine the columns, and therir new values. by using the columnId, and set the new value in each mixin column. * Resposen * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data: empty for now. ### 4.3 Remove Mixin Delete Mixin value for specific node. **DELETE** `/v1/{projectId}/mixinValue` [IntegrationAPIs\_swagger.json](/openapi.json) * Path parameters: * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Request body * targetNodeId: for what node you want to remove the mixin value(s). * mixinId: which mixin you want to remove. * mixinColumnIds: which column(s) to remove their value(s). ## 5. Bulk Operations ### 5.1 Bulk Create You can create nodes, relations, and mixins in one shot using this API. **POST** `/v1/{projectId}/bulkCreate` [IntegrationAPIs\_swagger.json](/openapi.json) * Path parameters * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Request body * nodes: (optional): array of nodes to be created: see [#create-node](#create-node "mention") * relations: (optional): array of relations to be created: see [#create-relation](#create-relation "mention") * mixins: (optional): array of mixins to be created: see [#create-mixins](#create-mixins "mention") * Response * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data: contains nodes cretae result, and relation create result * nodes create result: see [#create-node](#create-node "mention") * relation create result: see [#create-relation](#create-relation "mention") ## 6. Export JSON-LD ### 6.1 Fetch JSON-LD Import Profiles Returns the set of profiles that can be used in the export process. To export data from CoreModels as JSON-LD you need to use export profile. Using this API you can retrieve all import profiles you have for later use in the export operation. ## GET /v1/{projectId}/fetchJsonLdImportProfiles > ```json {"openapi":"3.1.1","info":{"title":"Integration API","version":"v1"},"security":[{"Bearer":[]}],"components":{"securitySchemes":{"Bearer":{"type":"apiKey","name":"Authorization","in":"header","description":"JWT Authorization header using the Bearer scheme. Enter 'Bearer' [space] and then your token in the text input below."}},"schemas":{"ObjectApiResponse":{"type":"object","properties":{"success":{"type":"boolean"},"error":{"$ref":"#/components/schemas/ErrorDetail"},"data":{}},"additionalProperties":false},"ErrorDetail":{"type":"object","properties":{"isFatal":{"type":"boolean"},"message":{"type":"string"}},"additionalProperties":false}}},"paths":{"/v1/{projectId}/fetchJsonLdImportProfiles":{"get":{"tags":["IntegrationApi"],"parameters":[{"schema":{"type":"string"},"in":"path","name":"projectId","required":true}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ObjectApiResponse"}},"application/json":{"schema":{"$ref":"#/components/schemas/ObjectApiResponse"}},"text/json":{"schema":{"$ref":"#/components/schemas/ObjectApiResponse"}}}}}}}}} ``` * Path parameters * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Response * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data: list of objets each object consists of * Id: profile Id * label: profile name (label). ### 6.2 Export JSON-LD Export types, and/or elements, and/or taxonomies from a certain space using a specific configuration profile. ## GET /v1/{projectId}/exportJsonLd > ```json {"openapi":"3.1.1","info":{"title":"Integration API","version":"v1"},"security":[{"Bearer":[]}],"components":{"securitySchemes":{"Bearer":{"type":"apiKey","name":"Authorization","in":"header","description":"JWT Authorization header using the Bearer scheme. Enter 'Bearer' [space] and then your token in the text input below."}},"schemas":{"StringApiResponse":{"type":"object","properties":{"success":{"type":"boolean"},"error":{"$ref":"#/components/schemas/ErrorDetail"},"data":{"type":"string"}},"additionalProperties":false},"ErrorDetail":{"type":"object","properties":{"isFatal":{"type":"boolean"},"message":{"type":"string"}},"additionalProperties":false}}},"paths":{"/v1/{projectId}/exportJsonLd":{"get":{"tags":["IntegrationApi"],"parameters":[{"schema":{"type":"string"},"in":"path","name":"projectId","required":true},{"schema":{"type":"string"},"in":"query","name":"spaceId"},{"schema":{"type":"string"},"in":"query","name":"configTypeId"},{"schema":{"type":"boolean","default":true},"in":"query","name":"exportTypes"},{"schema":{"type":"boolean","default":false},"in":"query","name":"exportElements"},{"schema":{"type":"boolean","default":false},"in":"query","name":"exportTaxonomies"},{"schema":{"type":"boolean","default":false},"in":"query","name":"exportDataTypes"},{"schema":{"type":"boolean","default":false},"in":"query","name":"includeSpace"},{"schema":{"type":"array","items":{"type":"string"}},"style":"form","explode":true,"in":"query","name":"nodeIds"},{"schema":{"type":"boolean","default":true},"in":"query","name":"graphBased"},{"schema":{"type":"boolean","default":false},"in":"query","name":"useDefaultType"}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/StringApiResponse"}},"application/json":{"schema":{"$ref":"#/components/schemas/StringApiResponse"}},"text/json":{"schema":{"$ref":"#/components/schemas/StringApiResponse"}}}}}}}}} ``` * Path parameters * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Query parameters * spaceId: determine the space Id to use in export process. * configTypeId: determine profile Id to use in export process, to get the full list of available profiles in a project see [#fetch-import-profiles](#fetch-import-profiles "mention") * exportTypes: (optional), default to false, if set to true types will be exported. * exportElements: (optional), default to false, if set to true elements will be exported. * exportTaxonomies: (optional) default to false, if set to true taxonomies will be exported. * includeSpace: (optional) include space info in the exported result, the space will be added in a special key `CoreModels_Space`. * graphBased: (optional - default is true) controls whether the export is using the `@graph` array or will be exporting a single node * useDefaultType: (optional) controls whether the export will automatically add `rdfs:Class` for types and `rdfs:Property` for elements * Response * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data: the exported Json-Ld data ## 7. Merge JSON-LD Imports new updates into a CoreModels space, this API triggers the same merge process that can be done through the user interface, you can add a URL or the github info. * Path parameters * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Query parameters * SpaceId: The space id that will be imported into, to get the spaces in a project, refer to [#fetch-nodes-1](#fetch-nodes-1 "mention"), and use NodeType='Space'. * ConfigTypeId: the configuration profile Id that the merge process will use to map the JSON-LD file properties into to CoreModels mixins and relations, to get the full list of available profiles in a project see [#fetch-import-profiles](#fetch-import-profiles "mention") * SourceDtos: an array of the sources, you can either add the github info or the file url. * Owner: The github repository owner of the source file. * Repository: The github repository name of the source file. * Branch: The github repository branch name of the source file. * FilePath: The target file path in the github repositry. * FileUrl: The target jsonld file absolute url. * OverrideNewPropertiesWarning (optional flag):\ If the github file had unmapped properties, CoreModels will return an error, this flag will skip the error.\ Note: this will happen if the file is not compatible with the configuration profile, we recommend creating a new import profile or manually verifying the profile. * OverrideDifferentSourceWarning (optional flag):\ CoreModels will verify that the space was originally linked to the provided information, and will return an error if the arguments referred to different source, this flag will skip this error. * OverrideOverwriteWarning (optional flag):\ CoreModels will check if any local changes in the space, and return an error (so the user wouldn't overwrite the changes) this flag will skip this error. * OnlyAddAndUpdate (optional flag):\ When using this flag, CoreModels will only add new nodes and update existing nodes (according to how they match the current schema), but it will not remove any node from the space. * Response * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data: the status of the import response. * Example:\ if we are synchronizing the following file \ We would use the following github information: * Owner: ModarAlNacharA * Repository: my-sample-repository * Branch: main * FilePath: sample-folder/JSON-LD sample file.json * Alternativly: we could use the raw url provided by github as the FileUrl: ![](/docs-assets/xA1CNs8xIRL2GKt8dJE1.png) ## 8. Export JSON Schema ### 8.1 Fetch JSON Schema Import Profiles Returns the set of profiles that can be used in the export process. To export data from CoreModels as JSON Schema you need to use export profile. Using this API you can retrieve all the JSON Schema import profiles you have for later use in the export operation. ## GET /v1/{projectId}/fetchJsonSchemaImportProfiles > ```json {"openapi":"3.1.1","info":{"title":"Integration API","version":"v1"},"security":[{"Bearer":[]}],"components":{"securitySchemes":{"Bearer":{"type":"apiKey","name":"Authorization","in":"header","description":"JWT Authorization header using the Bearer scheme. Enter 'Bearer' [space] and then your token in the text input below."}},"schemas":{"ObjectApiResponse":{"type":"object","properties":{"success":{"type":"boolean"},"error":{"$ref":"#/components/schemas/ErrorDetail"},"data":{}},"additionalProperties":false},"ErrorDetail":{"type":"object","properties":{"isFatal":{"type":"boolean"},"message":{"type":"string"}},"additionalProperties":false}}},"paths":{"/v1/{projectId}/fetchJsonSchemaImportProfiles":{"get":{"tags":["IntegrationApi"],"parameters":[{"schema":{"type":"string"},"in":"path","name":"projectId","required":true}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/ObjectApiResponse"}},"application/json":{"schema":{"$ref":"#/components/schemas/ObjectApiResponse"}},"text/json":{"schema":{"$ref":"#/components/schemas/ObjectApiResponse"}}}}}}}}} ``` * Path parameters * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Response * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data: list of objets each object consists of * Id: profile Id * label: profile name (label). ### 8.2 Export JSON Schema Export types, and/or elements, and/or taxonomies from a certain space using a specific configuration profile. ## GET /v1/{projectId}/exportJsonSchema > ```json {"openapi":"3.1.1","info":{"title":"Integration API","version":"v1"},"security":[{"Bearer":[]}],"components":{"securitySchemes":{"Bearer":{"type":"apiKey","name":"Authorization","in":"header","description":"JWT Authorization header using the Bearer scheme. Enter 'Bearer' [space] and then your token in the text input below."}},"schemas":{"StringApiResponse":{"type":"object","properties":{"success":{"type":"boolean"},"error":{"$ref":"#/components/schemas/ErrorDetail"},"data":{"type":"string"}},"additionalProperties":false},"ErrorDetail":{"type":"object","properties":{"isFatal":{"type":"boolean"},"message":{"type":"string"}},"additionalProperties":false}}},"paths":{"/v1/{projectId}/exportJsonSchema":{"get":{"tags":["IntegrationApi"],"parameters":[{"schema":{"type":"string"},"in":"path","name":"projectId","required":true},{"schema":{"type":"string"},"in":"query","name":"configTypeId"},{"schema":{"type":"string"},"in":"query","name":"spaceId"},{"schema":{"type":"string"},"in":"query","name":"rootNodeId"},{"schema":{"type":"boolean","default":true},"in":"query","name":"useAnchorForRef"}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/StringApiResponse"}},"application/json":{"schema":{"$ref":"#/components/schemas/StringApiResponse"}},"text/json":{"schema":{"$ref":"#/components/schemas/StringApiResponse"}}}}}}}}} ``` * Path parameters * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Query parameters * spaceId: determine the space Id to use in export process. * configTypeId: determine profile Id to use in export process, to get the full list of available profiles in a project see [#id-8.1-fetch-import-profiles](#id-8.1-fetch-import-profiles "mention") * rootNodeId: the root node id to use for export operation, this is the node that will be used as the root of the JSON Schema. * useAnchorForRef: if true, the exported JSON Schema will use the `$anchor` if available, other wise it will always use the JSON Path when referencing JSON Schemas in the `$ref` - default true * Response * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data: the exported Json schema data ## 9. Merge JSON Schema Imports new updates into a CoreModels space, this API triggers the same merge process that can be done through the user interface, you can add a url or the github info. * Path parameters * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Query parameters * SpaceId: The space id that will be imported into, to get the spaces in a project, refer to [#fetch-nodes-1](#fetch-nodes-1 "mention"), and use NodeType='Space'. * ConfigTypeId: the configuration profile Id that the merge process will use to map the JSON-Schema file properties into to CoreModels mixins and relations, to get the full list of available profiles in a project see [#id-8.1-fetch-import-profiles](#id-8.1-fetch-import-profiles "mention") * SourceDtos: an array of the sources, you can either add the github info or the file url. * Owner: The github repository owner of the source file. * Repository: The github repository name of the source file. * Branch: The github repository branch name of the source file. * FilePath: The target file path in the github repositry. * FileUrl: The target jsonld file absolute url. * OverrideNewPropertiesWarning (optional flag):\ If the github file had unmapped properties, CoreModels will return an error, this flag will skip the error.\ Note: this will happen if the file is not compatible with the configuration profile, we recommend creating a new import profile or manually verifying the profile. * OverrideDifferentSourceWarning (optional flag):\ CoreModels will verify that the space was originally linked to the provided information, and will return an error if the arguments referred to different source, this flag will skip this error. * OverrideOverwriteWarning (optional flag):\ CoreModels will check if any local changes in the space, and return an error (so the user wouldn't overwrite the changes) this flag will skip this error. * OnlyAddAndUpdate (optional flag):\ When using this flag, CoreModels will only add new nodes and update existing nodes (according to how they match the current schema), but it will not remove any node from the space. * Response * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data: the status of the import response. * Example:\ if we are synchronizing the following file \ We would use the following github information: * Owner: ModarAlNacharA * Repository: my-sample-repository * Branch: main * FilePath: sample-folder/SampleJsonSchema.json * Alternativly: we could use the raw url provided by github as the FileUrl: ## 10. Validate JSON Schema Validates a json string against a specific JSON Schema. ## GET /v1/{projectId}/validateJSON > ```json {"openapi":"3.1.1","info":{"title":"Integration API","version":"v1"},"security":[{"Bearer":[]}],"components":{"securitySchemes":{"Bearer":{"type":"apiKey","name":"Authorization","in":"header","description":"JWT Authorization header using the Bearer scheme. Enter 'Bearer' [space] and then your token in the text input below."}},"schemas":{"JsonSchemaValidationResultApiResponse":{"type":"object","properties":{"success":{"type":"boolean"},"error":{"$ref":"#/components/schemas/ErrorDetail"},"data":{"$ref":"#/components/schemas/JsonSchemaValidationResult"}},"additionalProperties":false},"ErrorDetail":{"type":"object","properties":{"isFatal":{"type":"boolean"},"message":{"type":"string"}},"additionalProperties":false},"JsonSchemaValidationResult":{"type":"object","properties":{"isValid":{"type":"boolean"},"jsonSchemaValidationErrors":{"type":"array","items":{"type":"string"}}},"additionalProperties":false}}},"paths":{"/v1/{projectId}/validateJSON":{"get":{"tags":["IntegrationApi"],"parameters":[{"schema":{"type":"string"},"in":"path","name":"projectId","required":true},{"schema":{"type":"string"},"in":"query","name":"configTypeId"},{"schema":{"type":"string"},"in":"query","name":"jsonString"},{"schema":{"type":"string"},"in":"query","name":"spaceId"},{"schema":{"type":"string"},"in":"query","name":"rootNodeId"}],"responses":{"200":{"description":"OK","content":{"text/plain":{"schema":{"$ref":"#/components/schemas/JsonSchemaValidationResultApiResponse"}},"application/json":{"schema":{"$ref":"#/components/schemas/JsonSchemaValidationResultApiResponse"}},"text/json":{"schema":{"$ref":"#/components/schemas/JsonSchemaValidationResultApiResponse"}}}}}}}}} ``` * Path parameters * projectId: see [#fetch-nodes](#fetch-nodes "mention") * Query parameters * configTypeId: determine profile Id to use in export process, to get the full list of available profiles in a project see [#id-8.1-fetch-import-profiles](#id-8.1-fetch-import-profiles "mention") * jsonString: the json that needs to be validated against the schema. * spaceId: (optional) The Space where the schema exists. * rootNodeId: (optional) the node that represents the JSON Schema. (use either the spaceId or the rootNodeId) * Response * success: see [#fetch-nodes](#fetch-nodes "mention") * error: see [#fetch-nodes](#fetch-nodes "mention") * data: * IsValid: if the json string is valid against the schema * JsonSchemaValidationErrors: the validation errors --- # Core Concepts Source: https://learn.coremodels.io/concepts Welcome to the Core Concepts page. These are the fundamental building blocks of a schema in CoreModels. Understanding these concepts is key to designing, visualizing, and managing your data structures effectively. *** #### Types & Elements At the heart of your schema is the Type, which serves as a blueprint or template for a data structure. A Type defines the kind of information that can be stored, such as a `Product` or a `Blog Post`. Each Type is composed of Elements, which are the individual fields that hold the actual data. For example, a `Product` Type would have Elements like `ProductName` and `Price`. * In SQL, Types are like the tables and Elements are like the Columns. * In Object Oriented programming languages, types are the classes, and Elements are the properties. * In Forms, the Type is the whole form, and the Elements are the fields inside that form. * In Content Management Systems (CMSs). All content types are Types and the fields are the elements, For example a `Blog Post`, and the fields for `Title`, `Body`, and `Author` are the Elements. Here are a few more examples: ![Examples of Properties/Elements](/docs-assets/wuNpqrjI6cohRtbt5hkZ.png) *Examples of Properties/Elements* *** #### Taxonomies Taxonomies are classification systems that help you organize your data into categories. They can be a simple, flat list of options (like a dropdown menu) or a nested, hierarchical structure with parent and child categories. You can link a Taxonomy to an Element to ensure users select from a predefined list of choices, guaranteeing data consistency. For example, a field in a registration form, could require the user to select the value for the `Job Title` field from a fixed set of options: `Developer`, `Product Owner`, `Designer` etc.. *** #### Mixins Mixins are a flexible way to add metadata or attributes to your `Types`, `Elements`, or `Taxonomies`. They don't contain data themselves but provide information or rules about the data structure. Think of them like C# attributes or setting a "nullable" checkbox in a database, or any validation rules that goes around a field. For example, an `Email` Element in a `Form` can have a "required" Mixin to mark it as a mandatory field, it also has a special validation (format) to insure having the correct structure for a valid email address. *** #### Relations Relations define the connections between different Nodes (Types, Elements, Taxonomies) in your schema. They allow you to link one piece of data to another, reflecting real-world connections. A Relation establishes a clear and structured link from an Element in one Type to an entire record in another Type. For example, a `Book` Type could have a `BookAuthor` Element that has a relation to a separate `Author` Type. A `Scientific Article` can inherit all the elements of `Article` to reduce duplications and promote reusability. Other examples of relations could be: ![Examples of Relations](/docs-assets/gc76ipUbfEWbI5xy8X0s.png) *Examples of Relations* *** #### Spaces A Space is the top-level container that holds all the different parts of a project, schema, or content repository. It's a complete, self-contained environment that organizes your Types, Elements, Taxonomies, and other components. Think of a Space like a dedicated folder or a single application where all the data structures and content for one specific project live. but unlike a simple folder, assets within your system -such as a Type- can belong to more than one Space. This allows for reusability without duplication. a Space is the technical term for a single schema, and the Core Data Model is the unified structure that is built by connecting multiple Spaces together. Read more at the [Core Data Model](/concepts/core-data-model). *** ### Terminology notes: #### Nodes & Relations Since the Types, Elements, Taxonomies and the other CoreModels components are interconnected in a dynamic way, far more agile than Tables or Spreadsheets, they are easier imagined and referred to as *Nodes* in a graph, that are connected by the *relations*. \ So for ease of reference, we refer to Types, Elements, Taxonomies, and any other CoreModels structure that can be connected using relations as **Nodes** that floats in the schema graph**.** #### **Schema.org terms and concepts** In CoreModels, we adopt some terms from the schema.org catalog: * 'SubClassOf' → '*Inheritance*' Concept (is a).\ Example: a `car` is a `vehicle`, so it inherits all the common properties of a vehicle. * 'DomainIncludes' → '*Composition*' (has-a).\ Example: a `car` has a `transmission` * 'RangeIncludes' → '*Applicability*' (can-be).\ Example: a `transmission` can be `manual` or `automatic` --- # Core Data Model Source: https://learn.coremodels.io/concepts/core-data-model Understanding CoreModels' data modeling methodology begins with a solid grasp of what a data model is and how our product helps you build a **Core Data Model**. This page outlines our approach to creating a unified schema for your entire ecosystem. *** #### What is a Data Model? A Data Model is an abstract framework that organizes data entities within a system, outlining their properties and interconnections. The concept exists in many fields: * In SQL, it refers to the collection of tables, columns, views and their relations that define how the data is stored in a certain database. * In Programming, it's the equivalent of a design diagram of all the classes and their connections. * In a web page, it's the underlying structure of a website (header, footer, navigation, forms, and other visual components). In CoreModels, all of these ideas can be represented through nodes (types, elements, taxonomies, etc..) and their interconnected relations. These nodes exist within a defined *space*. You can think of the space as the technical term used in CoreModels to refer to a schema of a single system. We recommend using a different space for each data model in your ecosystem. ![Examples of Data Models](/docs-assets/MfObmq2I3MiNKKw1x0rT.png) *Examples of Data Models* *** #### The CoreModels Methodology CoreModels helps you build a **Core Data Model**, which is a unified, inclusive structure that documents and links entities from various data models of different systems, standards, or teams. This approach: * Standardizes Workflows: By defining a single, shared model, you ensure consistency across different projects and teams. * Bridges Silos: It connects disparate data sources, breaking down the barriers between systems and making all your data visible in one place. * Ensures Governance: A central model simplifies data quality management and provides a single location for governance rules. ![An example of a Core Data Model](/docs-assets/KXNw0JWDNm2OuxH6b0cI.png) *An example of a Core Data Model* *** #### The Role of Spaces A Space is the technical term used in CoreModels to refer to a single schema. In business terms, you can think of a Space as a container for a specific project's data, such as a "Product Catalog Schema" or a "User Profiles Schema." A single project can be composed of multiple Spaces, each representing a different schema. These Spaces are interconnected, allowing you to link data between them. *** #### Building a Core Data Model In CoreModels, you build a Core Data Model by linking multiple Spaces together. This approach allows you to break down a complex system into smaller, manageable schemas while still maintaining a single, unified view of your entire data. The way you organize your Spaces depends on the complexity and scale of your project. **Example 1: One Space per System** For a business with multiple distinct systems, a single Space can be used to contain the schema for each system. For example, a company might have a separate Space for their `ERP System`, a Space for their `CRM`, and a Space for their main `E-commerce` project. This works well when each system's schema is of a manageable size. **Example 2: Multiple Spaces for a Single Project** For very large or complex projects, it is more effective to use multiple Spaces for a single project. The schema for each major module can be given its own Space. For example, a complete e-commerce project might contain a `Product` Space, a `Customer` Space, and an `Orders` Space. By creating relations between these individual schemas, you build a comprehensive Core Data Model for the entire business. This flexibility allows you to organize your schemas at the level that best suits your team and the complexity of your project. Your Next Step: Let's get started by [Building your first Schema (Part 1)](/getting-started/first-schema) If you already have your schema available in JSON-LD or JSON Schema, you can follow the guides for importing those schemas, [JSON-LD](/guides/json-ld) --- # FAQ Source: https://learn.coremodels.io/faq ### What's the default relation when creating Types and Elements ? The default relation used by CoreModels when creating a Type along with its elements is the 'Domain Includes' relation which is the equivalent of (has-a) or the 'Composition' concept in programming languages ### When should I use 'Domain Includes'/'Composition' and when to use 'SubClass Of'/'Inheritance' ? Use *SubClassOf* when it’s an “is-a” relationship * You’re saying one thing is a specialized type of another. * All properties of the parent apply to the child. Examples: * `Car` subClassOf `Vehicle` (every Car is a Vehicle). * `ScientificPost` subClassOf `BlogPost`. Use this when the type is just a more specific version of another.![](/docs-assets/vNxhTEXfHub5UIIL8Han.png) *** Use *DomainIncludes* when it’s a “has-a” relationship (composition) * You’re saying: this property belongs to this class. * It defines where a property can appear. Examples: * `Transmission` domainIncludes `Car`.\ → “Cars have transmissions.” * `Author` domainIncludes `Article`.\ → “Articles have authors.” Use this when you’re attaching properties (features, attributes, parts) to a class.![](/docs-assets/m6hkPIRh3KLZc7deK0Gl.png) ### What is the difference between a Project and a Space? A Project is the highest-level container in CoreModels. It holds one or more Spaces and defines the overall team and permissions. A Space is the technical term for a single schema and contains all the Types and Elements that make up that schema. Think of a Project as a portfolio and a Space as a single, organized project file within it. ### Schemas, Spaces, and Data Models, what is the difference ? Schemas, Spaces, and Data Models are three interconnected concepts that describe the structure of your data at different levels of a project. #### Data Model A Data Model is the abstract blueprint or framework that defines how all your data is organized. It's the highest-level concept, outlining the entities, their properties, and their relationships. A single Data Model can represent your entire business's data ecosystem. Think of it as a complete architectural plan for a building. *** #### Schema A Schema is the technical implementation of a Data Model. It's the specific, formalized set of rules that governs the structure of your data. While a Data Model is the abstract plan, a Schema is the code that defines it, such as a JSON Schema file or a set of SQL table definitions. A single Data Model can have different Schemas depending on the system or format used. *** #### Space A Space is the technical term in CoreModels for a single Schema. It acts as a project-level container that holds all the Types, Elements, and other assets that belong to a single, defined schema. A single project can have multiple Spaces, with each Space representing a distinct schema. ### How can I add rules and constraints to my schema ? You can add rules and constraints to your Elements using Mixins. A Mixin is a small piece of metadata that defines a specific behavior or rule. For example, you can add a `required` Mixin to an Element to make it mandatory, or a `minLength` Mixin to enforce a minimum number of characters. ### How do I collaborate with my team ? CoreModels is built for collaboration. First, you need to invite others to your project, open the 'share' menu that is available on the top right, then type your team member's email and select their role.\ When clicking the 'Add User' button they will receive an email if they doesn't have an account already. To collaborate with your team or even only to add notes, you can right click on any cell in CoreModels grids and click on 'Add Comment', which will start a thread linked to this cell. ### Can I add My own relations ? Yes you can add any relation to CoreModels by following these steps: 1. Open the 'Features Tray' (by clicking the hamburger menu (☰) under the CoreModels logo) 2. Click on 'Relations' → Plus button 3. Add a name for the relation, and specify the 'from' and 'to' parts 4. Click 'Add' Now you can configure the grids to show the relation and use it. **Need more help?** If you have any questions that are not answered in our documentation, please contact us at `tech@schematica.io` or use the 'Report an Issue' feature directly in the UI. --- # Building your first Schema (Part 1) Source: https://learn.coremodels.io/getting-started/first-schema ### Your First Schema: Building a 'Blog Post' Type Welcome to your first hands-on guide with CoreModels. In this tutorial, we'll walk you through building a simple `Blog Post` schema from scratch. You'll use the core concepts you've learned to create your first Type and its Elements. *** #### Step 1: Log In / Create an account Before we begin, you need to log in to your account. 1. Navigate to the [go.coremodels.io](https://go.coremodels.io/) login page. 2. Click "Login to CoreModels" and use your preferred google or microsoft account. *** #### Step 2: Create a New Project In CoreModels, a "model" is the same as a Project. It's the highest-level container for all your schema assets. 1. From the left pane, click on "New Model". 2. Give your Project a clear name, like "My First Blog Project", and click "Create". You now have a clean project to work in. *** #### Step 3: Create a New Space (optional) Now you need a Space to house your new schema. 1. Inside your new Project, from the top left dropdown (next to the CoreModels logo) click on the "All Spaces" drop down and click on "Add". 2. Give your Space a clear name, like "My First Blog Schema", and click "Create". You now a space to categorize your nodes in. *** #### Step 4: Define Your First Type with its Elements Next, you'll create a Type to serve as the blueprint for all your blog posts. 1. In the Types grid UI, click on the green plus icon, it will open the 'Create New Type' modal. 2. Name your Type `Blog Post`. 3. (optional) if you created a space, then you can select it. 4. Click on the drop down next to 'Create', and click on the first option 'Create and Add Elements'. 5. Name the first element as `Title` 6. Click 'Add and Close' You have now created a type and its first element. Note that the relation `Blog Post` → `Title` is called 'Domain Includes' Checkout the FAQ section for more information, [FAQ](/faq#whats-the-default-relation-when-creating-types-and-elements). *** #### Step 5: Add the rest of the Elements Now, let's add the rest of the fields that a blog post needs. These are the Elements that will hold your content. 1. Within the `Blog Post` Type you just created, expand the "Elements" column. 2. Click on the caret icon next to the plus button, and click on 'Create Element' 3. Create the following Elements: * Name: `Summary` * Name: `Body` Your `Blog Post` Type now has a `Title`, a `Summary`, and a `Body` field. *** #### Step 6: Add a Mixin for a Rule Let's make sure every blog post has a title by adding a rule. First we need to create the mixin 1. Click on the hamburger menu (☰) under the CoreModels logo, to open up the features tray. 2. Click on 'Mixins' (scroll down the features tray if it doesn't show up) and click on the plus button. 3. Set the mixin name as 'Validation Rules Mixin' 4. Click 'Create and Add Elements' 5. Set the mixin element name as 'Required' and choose the data type as 'Boolean', click Add & Close Now let's use the mixin for our `Title` field 1. In the 'Types Grid' on the far right, you'll find the sidebar tab, click on 'configuration', which will expand the configuration menu. 2. In the 'Elements Mixins' select the 'Validation Rules Mixin', and close the configuration by re-clicking on the sidebar. 3. The column 'Required' should show up in the Elements grid. 4. Click on the checkbox, for the `Title` field to mark the `Title` as required You've just added a rule to your schema that will prevent any blog post from being saved without a title. *** #### What's Next? Congratulations! You've successfully created your first schema. In just a few minutes, you've used Spaces, Types, Elements, and Mixins to build a functional blueprint. From here, you can continue to expand your schema by: * Adding a Taxonomy for `Categories` to organize your posts. * Creating a Component to display your blog posts on a web page. * Saving a sample record as an Exemplar for documentation. --- # Relations and Taxonomies (Part 2) Source: https://learn.coremodels.io/getting-started/relations-and-taxonomies In this section we'll learn about the default relations, create another type `Scientific Post`, and a taxonomy for the `StudyType`. *** #### Step 1: Inheritance Let's start by Creating the `Scientific Post` type, but since this is a 'Post' type which shares all the elements that already existed in the `Blog Post`, we can simply 'Inherit' this type. 1. Open the 'Create New Type' modal. 2. Type in the type name `Scientific Post`, and choose the `Blog Post` in the SubClass Of dropdown. 3. Now click on 'Create and Add Elements' 4. Add the `StudyType`, which is a unique element to this type You now have created a type and learnt about the inheritance relation or 'SubClass Of' as in the Schema.org terms. *** #### Step 2: Create a new taxonomy Let's create a new taxonomy to limit the options of the `StudyType`. 1. From the feature tray, click on 'Taxonomies', then click on 'Taxonomies Grid'.\ (You'll notice that it'll open up a new grid in a new CoreModels tab for the Taxonmies) 2. Click on the plus icon, and type in `Study Type Taxonomies`. 3. Click on 'Create and Add Child'. 4. Add the following Taxonomies as children for the `Study Type Taxonomies`: 1. `Experimental` 2. `Observational` 3. `Theoretical` You now have created a taxonomy with its children *** Step 3: Link the taxonomy to the element using the 'Controlled list' relation. 1. Open the Types Grid (or simply navigate back to it, by clicking on the CoreModels Tab 'Types Grid') 2. Open the 'Configuration' Side panel (by clicking the vertical configuration button on the far right) 3. Select 'Controlled List' from the 'Elements Columns' dropdown 4. Expand or open the Elements grid at the `Scientific Post` row (by clicking either icons in the 'Elements' column next to the type) 5. Double click on the 'Controlled List' cell (at the `StudyType` row under the 'Controlled List' column) 6. Add the taxonomies `Experimental`, `Observational`, and `Theoretical` from the dropdown. Now the element is linked to the taxonomies. Optional: Click on the `Scientific Post` type to see the type card with all the relations and elements, you can click on the nodes in the graph or change the visualization 'Tree (Dendrogram)' to explore more options\ check out the 'Hierarchical (Dagre)' option. --- # Crash Course Source: https://learn.coremodels.io/getting-started/crash-course Reviewing the [Terminology 101](https://vimeo.com/user45806891/cm-v2-2-crash-course-core-concepts?share=copy) can help provide additional understanding of key concepts (\~12 min). * [Terminology - What is a Type?](https://vimeo.com/user45806891/cm-v2-2-crash-course-core-concepts?ts=88000\&share=copy) * [Terminology - What is a Property/Element?](https://vimeo.com/user45806891/cm-v2-2-crash-course-core-concepts?ts=107000\&share=copy) * [Terminology - What are Taxonomies / Lists / Enums?](https://vimeo.com/877042475/fe7b213ad2?ts=205000) * [Terminology - What is Mapping?](https://vimeo.com/877042475/fe7b213ad2?ts=298000) * [Terminology - What is an Attribute?](https://vimeo.com/877042475/fe7b213ad2?ts=367000) * [Terminology - What are Mixins?](https://vimeo.com/877042475/fe7b213ad2?ts=417000) * [Terminology - What are Relations?](https://vimeo.com/877042475/fe7b213ad2?ts=241000) * [Content Modeling: What are Views?](https://www.youtube.com/watch?v=LGVwEoCMaXU) > **Note** > Module 1 provides an essential introduction for business users who plan to collaborate with the team but do not need to work directly on the model development. * [**Module 1**](https://vimeo.com/user45806891/cm-v2-2-crash-course-module1?share=copy) **- High-Level Introduction to CoreModels, Graph Modeling Methodology and Benefits, Demo Preview. (\~11 min):** * Addressing Data Silos and Model Alignment * High-Level Introduction to CoreModels * Graph Modeling Methodology and Benefits * High-Level Demo Preview > **Note** > Module 2 serves as a quick start for users to start working in CoreModels * [**Module 2**](https://vimeo.com/user45806891/cm-v2-2-crash-coursemodule2?share=copy) **- Signing in, Working with Types and Elements, Inheritance, and Composition in Grids and Workbench (\~21 min):** * Sign in to CoreModels * Creating a new Model * Working with Types in Grids * Working with Elements in Grids * Visualizing in Cards * Managing Types and Elements in the Workbench * Working with Inheritance in Grids * Working with Composition in Grids * [**Module 3**](https://vimeo.com/user45806891/cm-v2-2-crash-course-module3?share=copy) **- Taxonomies and Relations (\~14 min):** * Working with Taxonomies * Creating Relations * Configuring Relations in Grids * [**Module 4**](https://vimeo.com/user45806891/cm-v2-2-crash-course-module4?share=copy) **- Mixins, Attributes, Spaces, and Mappings (\~40 min):** * Explaining Mixins and how to create simple mixins and attribute mixins. * Configuring Mixins in grids. * Providing examples of Mixins as Attributes. * Introducing the Mapping Process and Spaces. * Examples of Mapping entities using Relations and Mixins. * [**Module 5**](https://vimeo.com/user45806891/cm-v2-2-crash-course-module5?share=copy) **- Components, Exemplars, and In-Context Values (\~20 min):** * Explaining the Concept of Components * Working with Components in CoreModels * Working with Exemplars in CoreModels * Introducing the in-context configurations * [**Module 6**](https://vimeo.com/user45806891/cm-v2-2-crash-course-module6?share=copy) **- Contributors and Collaboration (\~8 min):** * Adding and Managing Contributors * Discussing and Collaborating in CoreModels * Configuring Slack Integration Settings * [**Module 7**](https://vimeo.com/user45806891/cm-v2-2-crash-course-module7?share=copy) **- Cards, Graphs, and Grids Management (\~13 min):** * Cards * Graphs and Visualization layouts * Creating and Managing Grids. * [**Module 8**](https://vimeo.com/user45806891/cm-v2-2-crash-course-module8?share=copy) **- Intro to Export, Import, and Integration Options (\~8 min):** * Plugins, Import, and Export Channels * Importing and Exporting a JSON-LD Model * Integration options and APIs. * [**Module 9**](https://vimeo.com/user45806891/cm-v2-2-crash-course-module9?share=copy) **- Model Settings, Change Log, Backup, and Restore (\~4 min):** * Exploring the model settings * History and Change log * Duplicating projects * Backup and Restore options * [**Module 10**](https://vimeo.com/user45806891/cm-v2-2-crash-course-module10?share=copy) **- Reporting Issues, Support and Feature Requests (\~2 min):** * Submitting support and feature requests. * Tracking the reported issues. **Related resources:** [Core Concepts](/concepts) --- # JSON-LD Source: https://learn.coremodels.io/guides/json-ld CoreModels supports importing schemas using JSON-LD, which offers a convenient way to enrich JSON files with required schemas, including those found in schema.org. To learn more about JSON-LD, refer to this [useful link](https://json-ld.org/), currently both [graph based JSON-LD](https://json-ld.org/spec/FCGS/json-ld/20130328#named-graphs) and non-graph based JSON-LD are supported. You can upload your schema using this importer. Simply attach your file, or provide a URL for your schema, or connect your github account and fetch schemas from accessible repositories. Accepted file formats include .json or .jsonld. There are two different import types, and we will explain each one of them. | | | | --- | --- | | [Dynamic](/guides/json-ld/dynamic-import) | Import the JSON-LD properties using the import/export profile as seen in the step by step guide. | | [Merge](/guides/json-ld/merge) | Import the JSON-LD properties based on how you configure them and detects any new changes. First, it will open the import/export profile then proceeds to the Merge Grid to display and prompt for the changes. | --- # Dynamic Import (JSON-LD) Source: https://learn.coremodels.io/guides/json-ld/dynamic-import ## Step By Step Guide (First make sure to create a space so we can import our file into it - as this is a pre-request for any import operation) ![Create Space Modal](/docs-assets/FWb52llb3K5XEdY70Cwq.png) *Create Space Modal* From the upper right corner, click on the three-dot menu and select 'Plugins' and click 'Open' next to the JSON-LD Importer to open the import modal. ![UI Guide: Plugins](/docs-assets/JfzfAVHLmEiFUp96VkQV.png) *UI Guide: Plugins* Once the modal shows up, you can give a name for the import, this will be used in the logs and as a name for the import/export profile. You can either add the Schema URL or upload a file (with the .jsonld or .json extension), or even connect your account to github and fetch the file from there. We will use a simple example, you can download it using [this link](https://drive.google.com/file/d/1HK_jbHpmHEXo0BizQMDt4E0EHuZXttUN/view?usp=sharing). The 'In Space' field is required, you must create a space to be able to select it here. ![Import JSON-LD modal](/docs-assets/tqpXWeBjKmJMiLiOq7c6.png) *Import JSON-LD modal* Note: we recommend importing to an empty space since the dynamic importer will consider everything as new and will only add the nodes, if you want to only update the schema using an external source, please refer to the [Merge](/guides/json-ld/merge) page. Once everything is set, click on 'Next'. CoreModels will then scan and validate the JSON-LD schema, then open the import/export profile UI with the suggested transformation. Here's how the profile looks like with the properties from the sample file ![](/docs-assets/e7MM9TakwWWzcXQyJpVD.png) 1. The properties from the file (re-ordering those will affect the export process) 2. Each property mapping 3. Profile Settings Coremodels will suggest the transformation based on the property name and type:\ Example1: if the property is called 'comment' and all the values in that property are found as a simple text, Coremodels will search for all existing mixins, if it finds an exact match, it will suggest it.\ Example2: if the scan determines that the property can be transformed into a relation (by determining that the type is an 'object array' - see 'Type' in the table below), Coremodels will also search the existing relation names and suggest it. Note that you can instruct Coremodels to use the configuration from an old import profile, Coremodels will do so, but it will always create a new profile. ## Import/Export Profile The following table explains each input of the import/export profile. | Field | Description | Notes | | --- | --- | --- | | Key | The name of each JSON key that was found in the JSON-LD source. | | | Transformed To | This field controls the options found in 'transformation value'.; In simple terms, when a field is storing regular text values, we choose 'mixin'.; an example from the sample file would be the 'rdfs:comment' property.; But if the value is actually referring to another node in the graph, then we should set it as 'Relation' (ex: 'subClassOf' property). | None -> do not import.; Mixin -> the values will be stored as text, allows choosing mixins under 'transformation value' column.; New Mixin -> the values will be stored as text, allows entering the new mixin name.; Relation -> the values will point to other nodes, allows choosing relations under 'transformation value' column (if the importer finds a relation to a non-existing node, it will automatically create it).; Relation -> the values will point to other nodes, allows entering the new relation name; Space -> special handler to control the space property name. | | Transformation Value | The options for this field are controlled by the 'Transformed To' field, you can select the mixin/relation name here. | we recommend choosing or creating a mixin/relation that has the same name as the property so it would be easier for users to navigate through the model.; | | Export Format | CoreModels will identify each property's type according to the JSON-LD schema values that were found for this property.; changing this has no impact during the import but it's stored to preserve how each property should look like in the export. | "prefix:key": null -> Null; "prefix:key": "myValue" -> Primitive; "prefix:key": ["myValue"] -> Primitive Array; "prefix:key" : {"@id":"myValue"} -> Object; "prefix:key": [{"@id":"myValue"}] -> Object Array | | Default Value | This property provides a default value to used in the export when there's no value. | If we are exporting ‘rdfs:comment’ , and some nodes does not have any value in that mixin, the default value will be used from here (ex: "TBD").; You can also input as *@@null* to provide a null value. | | Value Prefix | If a property has a value that consistently starts with the same prefix, you can provide this value in this field. Coremodels will ignore that prefix during the import process and re-add it during the export. | | | Is Reversed | A checkbox used to reverse the relations direction.; An appropriate example for reversing the relation is [domainIncludes](https://schema.org/domainIncludes) from schema.org, since this relation (in the JSON-LD schema) is stored in the element but it's actually saying that the type should have this element.; domainIncludes is suggested to be reversed by default in Coremodels. | Technically, if we have this:; `"@id": "bts:NodeA",`; `"rdfs:subClassOf":[{"@id" :"bts:NodeB"}]`; the subclassOf relation is from NodeA to NodeB, but if you set 'Is Reversed' to true,; Node B will be a subclass of Node A in Coremodels. | * Coremodels will ignore the following: * Any property that has 'Transformed To' set as “null”. * Any property that has a 'Transformed to' set to mixins or relation but does not have a value under 'transformation value'. * Any property that does not have a key (if the key was removed from the profile). * CoreModels will validate against the case of having two properties with the same transformation value. * CoreModels requires that the @id property exists and the "Transformed To" is set to mixin and the 'transformation value' is also set - this is used as an identifier for the merge importer. * Coremodels will also store all the values that are found under the `@context` in the JSON-LD namespaces grid (can be found from the features tray -> Taxonomies -> JSON-LD Namespaces) Once all the transformations are set you can click on Proceed to start importing. The task details screen will show up to show the progress of each step, then it will auto refresh the page when the import is done. The tasks grid can be found from the upper right corner -> three-dot menu -> Tasks. After all the Import is finished, you can check the Types, Elements, and Taxonomies in your space. Here is an example of the Types after importing the sample file. ![Types Grid - after importing the sample JSON-LD file](/docs-assets/fXhdDH47otrJDHKG9uxG.png) *Types Grid - after importing the sample JSON-LD file* See the [Export (JSON-LD)](/guides/json-ld/export) guide to see how to export your schema as JSON-LD from CoreModels ## @type property '@type' is a special property that determines how a JSON-LD object is imported as either a type, element, or taxonomy in CoreModels. This is based on how the schema.org importer is originally designed according to the [rdfs definition](http://www.w3.org/2000/01/rdf-schema). * If "@type" is set to "rdfs:Class", then CoreModels will consider it as a type. * `{ "@id": "sampleType1", "@type": "rdfs:class" }` * If "@type" is set to "rdf:property", then it will be considered as an element. * `{ "@id": "sampleElement1", "@type": "rdf:property"}` * If "@type" is set to "rdfs:Class" AND the "rdfs:subClassOf" has "schema:enumeration", then it's a taxonomy. * `{ "@id": "sampleTaxonomyParent",` \ `"@type": "rdfs:Class",`\ `"rdfs:subClassOf": [{ "@id" : "schema:enumeration" }],` * If "@type" is set to any previous taxonomy, then it's a child of that taxonomy. * `{ "@id": "sampleTaxonomyChild", "@type": "sampleTaxonomyParent"}` * if "@type" was not set, the default behaviour is for CoreModels to consider the object as a type. Read more on the @type property in [this page](/guides/json-ld/json-ld-type-property) --- # Export (JSON-LD) Source: https://learn.coremodels.io/guides/json-ld/export In this guide you will see how to create a JSON-LD profile manually, or simply just how to export your imported schema. If you are following the tutorial from [Dynamic Import (JSON-LD)](/guides/json-ld/dynamic-import) you can skip to [#step-2-use-the-profile-to-export-your-schema](#step-2-use-the-profile-to-export-your-schema "mention") *** Before starting, let's assume that you have already created some nodes, and you already added some mixins and relations, we will assume that you have used a mixin to store a title, and a description for your nodes. #### Step 1: Create the import/export profile 1. From the features tray, click on 'Grids' → click on the plus icon. 2. Select 'JSON-LD Import/Export Profile' as the Grid Type, and add an appropriate name.\ The profile UI will automatically open up. 3. On the bottom left corner, Click on 'Add Key' 4. Add the Key name that will show up in the exported file (for example: title) 1. The 'title' card will show up, with the 'Transformed To' set to None. 2. Set 'Transformed To' to 'Mixin', then set 'Transfomration Value' to the mixin that has the value of the title. 3. Set the 'Export Format' to 'String', and (optionally) set the 'Default Value' to 'Default title' 4. Click 'Save' 5. Repeat the process for the assumed 'Description' field, and for any other mixins or relations that you are using in your model 1. Note that Relations are usually exported as 'Object Array' - this is set in the 'Export Format' field. Now that the profile is set, let's see how to use it. *** #### Step 2: Use the profile to export your schema 1. Navigate to the 'Types Grid', In the 'Types Grid' tab and on the far right, you'll find the sidebar tab, click on 'export', which will expand the export menu, then select 'Custom JSON-LD'. 1. Alternatively you can open the dedicated export UI, Features Tray → Channels → Export → Custom JSON-LD 2. Select 'Types', 'Elements' at the 'Export Includes' dropdown. 3. Select the profile you created (or that was created automatically during the import) at the 'Import Configuration' dropdown, let's leave the default values for the other options. 4. Click on 'Process', then after the export is done, you can either download the file or click on 'Preview' to see the export results Now that you have exported your schema using the default options, let's explore what the other options mean. #### Export Options **Include HTML \