Agentic data lineage: designing evidence controls across MCP, OpenTelemetry and OpenLineage
Updated: Oct 19
BearingNode's reference implementation connects execution telemetry and data lineage. The important findings concern what those records mean, how they relate and whether the supporting evidence survives.
In short: agentic data lineage records which governed datasets an AI agent's actions read or wrote, captured where the data is actually accessed and linked to the execution trace. In BearingNode's reference implementation, each SQL statement an MCP tool executes is recorded as one OpenTelemetry span and one OpenLineage run, each carrying the other's identifier. Neither is derived from the other. On its own published scoring against eleven evidence requirements, the implementation meets two and partially meets nine; the gaps include parentage, identity and authority, retention, privacy and evidential basis.
This is not about exposing a lineage catalogue to agents as an MCP tool, which is a separate and useful pattern. It is about the reverse: recording the lineage of the data agents themselves touch.
Start with the evidence requirement
An agent calls a tool. The tool executes SQL. The request succeeds.
What should an organisation be able to establish afterwards?
Which work occurred? Which datasets does the record identify? How were those identities established? Can the surrounding context be recovered? Is missing evidence detectable?
These questions cross data governance, data engineering, AI platforms and assurance. They also cross standards.
In *Building Track 3*, we described joining execution telemetry and data lineage without deriving one from the other. The BearingNode Lab now tests a bounded implementation of that approach, written as eleven testable requirements and scored in a reproducible evidence matrix.
Where should agentic data access be instrumented?
At the point where SQL reaches the database—not at the protocol.
The reference implementation runs an unmodified, pinned postgres-mcp server (0.3.0) and instruments it by subclassing its own SqlDriver, at SqlDriver.execute_query.
That location matters. Of postgres-mcp's nine tools, only three take SQL as an argument. The other six—such as list_objects or analyze_db_health—build and execute SQL internally, and the protocol never sees it. A gateway, proxy or middleware reading tools/call would capture the generic run-this-SQL tool and silently miss every purpose-built one.
There is a second reason. A component that parses somebody else's traffic infers what happened. Inference is weaker evidence than a record emitted by the process doing the work.
One statement, one span, one run
For each statement, the implementation:
Reads any inbound trace context and opens a statement execution span beneath it.
Parses the SQL and reads any supplied lineage parentage.
Creates a lineage run identifier and records it on the span.
Emits an OpenLineage START event.
Executes the SQL.
Emits COMPLETE or FAIL.
Closes the span.
One tool call can execute several statements. Each therefore needs its own statement span and lineage run, rather than successive run references overwriting one attribute on the tool-call span.
START precedes SQL deliberately. In fail-closed mode, failure to record START can prevent execution. Failure to record COMPLETE is different: the database operation has already happened, so fail-closed can only withhold the result from the agent. Fail-open is the default; both are implemented, and the choice is a control decision, not a technical one.
Capture an event, recreate it on read: what the producer records as the call goes out, and how an auditor, SRE or governance analyst reconstructs it.
OpenTelemetry vs OpenLineage: what each record carries
The two records answer different questions, so they carry different things:
OpenTelemetry span | OpenLineage run event | |
|---|---|---|
Answers | How did this call perform? | What data did this work touch? |
Carries in the Lab | Execution timing and status, trace and span IDs, lineage.run_id, lineage.actor_status, and lineage.dropped when an event is lost or caller details are discarded | Run ID, job, parsed input and output datasets, and a trace-context facet holding the trace and span IDs |
Dataset names | No dataset attribute, by design (error text on span events may echo statement fragments) | Yes, as parsed intent |
Handling | Typically sampled and short-retained; the Lab exempts governed traces from sampling | Sent to the lineage store (not guaranteed under fail-open); retention is the store's property |
Does not establish | Which governed data was read | Who initiated the run, or on whose authority |
The OpenLineage event carries trace-context references back to the span; the span carries the lineage run ID. They are a link: each record holds the identifier of the other, and neither is reconstructed from the other. The lineage.* span attributes are the Lab's own and are not vendor-prefixed; do not copy them as if they were a standard.
Trace propagation and lineage parentage must remain distinct, too:
Trace context connects execution activity.
Lineage parentage references a caller's lineage run, where that run exists.
In the Lab, parentage travels in a vendor-namespaced MCP _meta key, nested beside—not merged into—the trace context:
"_meta": {
"traceparent": "00-4bf92f…-00f067…-01",
"io.bearingnode.lineage/parent": {
"parentRunId": "<UUID of the caller's own lineage run>",
"jobNamespace": "<caller's job namespace>",
"jobName": "<caller's job name>"
}
}Illustrative values. The key is ours, not a standard; the server-side reader is built, but no client sends the lineage key yet—including the Lab's own demo driver, which sends only `traceparent`.
Parentage does not authenticate the caller, establish delegated authority or prove that a referenced parent record survives.
What an OpenTelemetry span carries, what an OpenLineage event carries, what both carry, and what neither does.
Be precise about evidential basis
The implementation identifies datasets by parsing SQL before execution.
That establishes intent expressed in the statement—not every object actually accessed by the database.
Basis | What it describes |
|---|---|
Statement parsing | Intent expressed in query text |
Resolved engine plan | Work represented by the engine's plan |
Execution observation | Access observed during execution |
The implementation labels its records as parsed intent and states that completeness is not guaranteed.
This also qualifies a simple slogan such as "declared, not inferred". Emitting the record from the executing process is valuable, but capture location does not turn a parsed dataset list into observed database effect.
The receiving system matters as well. Accepting an event is not proof that a product preserves its qualifications or exposes them to an investigator.
Test the read-back and the reader's view—not just the ingestion response.
Test the lifecycle, not just the join
A valid identifier is necessary for correlation. It is not sufficient for reconstruction.
The referenced span must arrive. Both records must remain retrievable. Relevant context must survive for compatible periods.
The Lab's OpenTelemetry Collector keeps, in full, any trace carrying lineage.run_id, ahead of a probabilistic sampler for everything else. Verified end to end on 30 September 2026, a run of 100 queries plus setup—111 SQL statements in all—produced 111 lineage runs in Marquez, all completed, and 111 retained traces in Jaeger. That is count parity at a point in time, not proof of equal retention periods. The retention is trace-scoped, which has a cost: one governed span keeps its entire trace for as long as the trace store retains it.
A single manual completed-versus-killed test, still recorded as an open risk, shows the boundary. A completed statement exported its span afterwards. When the process was killed during execution, the START record remained, but the referenced span did not arrive.
A Collector policy can protect received traces from sampling. It cannot manufacture spans that were never exported, or establish long-term retention parity.
Likewise, a lineage store cannot report an event it never received without another source against which to reconcile.
A control therefore needs more than successful examples. It needs detectable failures and evidence of operation over time.
Does this apply beyond MCP?
The Lab's position is that the gap does; the demonstration does not.
As the Lab frames it, nothing in the gap depends on MCP. It arises for consumption events where a runtime actor initiates ad hoc access to governed data—a REST or GraphQL call, SDK-based access or a notebook as much as an MCP tool call. OpenLineage hangs ownership off a Job declared before anything runs, and an ad hoc actor has no declared Job to hang off. The claim does not extend to all ad hoc human use of BI tools.
The discriminator, in the Lab's terms, is whether the work was declared in advance. A scheduled agent with a declared job should already be served by existing lineage producers—an assumption the Lab records as untested and open to challenge. MCP is the forcing function, not the scope.
What is built and verified is MCP, over Postgres, through SQL.
Build on standards—and change proposals when the evidence changes
We have opened or continued threads in each community:
MCP: a lineage parent key in _meta, proposed for discussion as a Financial Services interest-group extension—not identity or authority (financial-services-interest-group#16, following the closed #2638).
OpenLineage: two proposed Run facets—a trace-context provenance pointer, built under a vendor prefix, and a reference to the caller, outlined but not yet emitted (OpenLineage#4484).
OpenTelemetry: the implementation keeps its own lineage.run_id; our request for a standard reference attribute was withdrawn on 17 October 2026 (semantic-conventions-genai#589). We do not ask for dataset names on spans.
These are proposals and experimental findings, not adopted conventions, and none of the October proposals has had a maintainer response yet.
The Lab runs on MCP SDK 1.x and protocol revision 2025-11-25. A probe shows the _meta key and trace injection behaving as designed on a small stand-in server built on SDK 2.3.0. The Lab's own server does not yet run on 2.x.
This distinction is central to our approach. The objective is not to generate standards changes. It is to establish where existing mechanisms suffice and where a well-supported gap remains.
Why BearingNode publishes the supporting work
We support open standards whether the implementing product is proprietary or open source.
But interoperability needs more than a shared vocabulary. Producers and consumers must preserve the meaning and qualifications of the records they exchange.
That is why the Lab publishes its requirements, implementation, evidence matrix and RAID register together.
Readers can examine the problem definition, reproduce the assessment, challenge assumptions and understand why decisions changed. The reviewed assessment—two requirements met and nine partially met—makes the remaining work visible rather than presenting a complete-looking graph as a complete solution.
We believe that is a useful contribution to stronger industry practice: not only sharing an integration, but sharing the means to assess it.
What to test in your estate
The implementation covers structured data reached through SQL in Postgres over MCP, using synthetic data. It does not establish coverage of document retrieval, spreadsheets, non-SQL languages or production security requirements. One span and one run per statement may also behave badly for long-running and streaming work; that is recorded as an open risk. And in the Lab, SQL text and database error text reach the lineage store unredacted—a privacy control any adoption must add.
For an adopting organisation, the practical questions are:
Where is each actual data-access path instrumented?
What evidential basis supports the dataset relationships?
Does the consumer preserve and display qualifications?
What happens when START, terminal events or spans are lost?
Are supporting records retained for compatible periods, and at what cost?
Which identity and authority facts remain unavailable?
Can assurance test the control continuously rather than inspect one successful interaction?
The aim is not simply to connect a trace to a lineage graph. It is to build an evidence chain whose meaning, operation and limitations can be tested.
Frequently asked questions
What is the difference between OpenTelemetry and OpenLineage for AI agents? OpenTelemetry records how a call executed and is usually sampled and short-retained. OpenLineage records which datasets a piece of work read or wrote. For agent data access, keep both and link them by identifier rather than deriving one from the other.
Why instrument at the database driver rather than an MCP gateway or proxy? Because many MCP tools build SQL internally. In postgres-mcp, six of nine tools never expose their SQL to the protocol, so a gateway reading tools/call misses them. A driver-level hook also emits evidence rather than inferring it from traffic.
Does the MCP `_meta` key carry caller identity? No. In the Lab it carries lineage parentage—a reference to the caller's own lineage run, where one exists. Identity and delegated authority are separate facts and remain an open gap.
Do lineage events show what the database actually accessed? Not in this implementation. Datasets come from parsing the SQL, which records intent expressed in the statement. Views, functions and engine behaviour can mean actual access differs.
Can I run it? Yes. The Lab is source-available, uses synthetic general-insurance data and runs locally with Postgres, Jaeger and Marquez. Every evidence-matrix row has a command that reproduces it; the matrix notes where a command shows code behaviour rather than the property itself.
Inspect and challenge the work: BearingNode Lab — requirements, evidence matrix, implementation and RAID register
Leadership companion: AI governance needs evidence of the data an agent used
Reviewed against BearingNode Lab edition 2026.10.2, October 2026.
About the author: Daniel Rolles is CEO and Founder of BearingNode, and one of the key authors of BearingNode's Data and Information Observability Framework. He has spent over 30 years in data, analytics and AI, including leading EY's analytics practice for financial services, and is the author of the OpenLineage proposal on lineage for runtime actor-initiated data interactions referenced in this article.
