aigov!=aio11y, part 2: Three tracks, one audit question
- Daniel Rolles
- Jul 5
- 7 min read
In part one I argued that the industry is conflating AI governance and AI observability, exactly as we conflated data governance and data observability in the 2010s — and I left you with the question that decides everything. An auditor, a regulator, or a data subject asks:
"What data did this AI system access, and where did it go?"
Every agentic AI deployment is on one of three tracks with respect to that question. This post is about the fork.
The room where I asked it
I recently attended a hyperscaler's AI governance session, and I want to start by being fair to it: the engineering was genuinely impressive. Every chat interaction captured as spans and traces — classic OpenTelemetry territory. Session metrics. Latency. Token usage. Tool invocation patterns. Full audit trails, dashboards over the top. A real observability stack, well built, framed as governance.
The demo showed agent interactions beautifully traced: every tool call a span, every span in a trace. So I asked a question.
What happens when the agent makes an MCP tool call out to a data source — a data warehouse, say?
Because that is not just a runtime event. The agent queried tables, transformed the results, and injected them into a prompt. An ad-hoc data pipeline was created and executed inside a chat session. The span records that the tool was called, and how long it took. It does not record:
which tables and columns were read
what happened to the data between query and prompt
which model and session the data ended up in
what the downstream graph looks like when that output is consumed by another agent, another user, another system
There was no answer. Not because the engineering is bad — because the framing never asked the question. When observability is a sub-feature of AI governance, the data flow the AI system just created is invisible.
That room was on Track 2. Let me lay out all three.
Track 1: Don't capture
Capture nothing. No spans, no lineage, no evidence layer. Policy documents and hope.
I would love to say this track barely exists, but most organisations deploying AI today are on it — often without knowing, because a vendor dashboard somewhere creates the impression of visibility. If part one resonated, you already know why this track fails: it is the 2010s data governance playbook replayed verbatim. There is nothing more to say about Track 1 except: check, honestly, whether you are on it.
The interesting fight is between the next two.
Track 2: Capture everything in OpenTelemetry
This is the sophisticated default, and it is the track the best-engineered deployments are on right now — including that hyperscaler room.
And to be clear about what Track 2 delivers, because it is a lot: spans and traces for every agent step. Model calls, token counts, tool invocations, loops, latency, cost. The OpenTelemetry GenAI semantic conventions now define agent, workflow and tool spans, and MCP-specific conventions capture method names, tool names, session and transport metadata. If your question is *"how is the system behaving?"* — runaway tool loops, context truncation, cost blowouts, latency regressions — Track 2 answers it brilliantly. OTel is exactly the right tool for that job.
Here is the problem. Track 2 is a local maximum. Organisations arrive here, look at the dashboards, and believe they are done — and it is precisely the quality of the engineering that makes the remaining gap invisible.
Put the audit question to Track 2:
"What data did this AI system access, and where did it go?"
Track 2 answers: *"the agent called the query_warehouse tool at 14:32, and it took 240 milliseconds."*
That is not an answer. That is an access record. And access records are not lineage. A log tells you a call happened. Lineage is a graph with semantics — datasets, transformations, dependencies, propagation. You cannot reconstruct one from the other.
"So enrich the span"
The Track 2 rebuttal writes itself: put the dataset details on the span. Add attributes for the tables read, the columns touched, the transformations applied. One standard, one pipeline, job done.
Three reasons that does not hold:
Retention asymmetry. Traces are operational exhaust: sampled, aggregated, aged out in days or weeks. Lineage is an audit asset: retained for years, queried long after the trace is gone. The moment your data-flow evidence lives inside your trace store, your audit trail inherits your sampling policy. Ask your compliance function how that conversation goes.
Payload sensitivity and cardinality. OpenTelemetry's own guidance says not to capture sensitive content on spans by default — the recommended production pattern is to store content externally and record a reference. The spec itself is telling you the span is the wrong home for data semantics. It carries a pointer; the substance belongs elsewhere.
Consumer ecosystems. The systems that need to consume data-flow evidence — catalogues, governance platforms, lineage graphs — speak OpenLineage, not OTLP. Flattening lineage into span attributes strands it in a place your data estate cannot see.
One more caveat for balance: the OTel GenAI and MCP semantic conventions are themselves still in Development status as of v1.41 — attribute names can change without a major version bump. Even the mature side of this problem is unfinished. The lineage side has barely started.
Track 3: Spans and traces in OTel, lineage events in OpenLineage
The event we are arguing about — an agent's MCP call to a data store — is one event with two natures.
It is a runtime event: an actor invoked a tool, in a session, with a latency and a cost. That is a span. OpenTelemetry territory, fired by the agent framework.
It is simultaneously a data lineage event: a job read datasets, transformed them, and produced an output that others consumed. That is a run event. OpenLineage territory, fired when data actually moves.
These are not two layers of one discipline. They are two disciplines meeting at a junction — and the MCP call to a data source is the first event that requires both to fire simultaneously. AI observability owns the actor: which agent, which model, which session, which decision. Data observability owns the flow: which datasets, which transformations, which downstream consumers. Neither alone answers the audit question. AI observability without lineage cannot say what data moved. Data observability without agent context cannot say who moved it, or why.
The join is the trace context. Fire an OpenLineage event at the MCP boundary carrying the trace ID of the span that triggered it, and the AI behaviour and the data flow it created become one joined-up evidence trail: which agent, via which tool, read which datasets, producing which output, informing which decision.
That trail has a name: decision lineage.
And conceptually, none of this is exotic. The agent is playing the role a job has always played in the lineage graph. The warehouse does not care that the pipeline was written in natural language at runtime — it saw a read, a transformation, and an output, same as any Spark job or dbt model.
Technically, though, there is a genuine modelling question — and it is worth being precise about it. OpenLineage's current model quietly assumes that lineage is *declaration-based*: the relationship exists before the event that records it. An ETL job knows its inputs and outputs before it runs. A VIEW's derivation from base tables is defined at DDL time. Even jobless structural lineage declares a relationship that is durable and pre-existing. The emitting system records a fact that was already true.
Agentic interactions invert that. The lineage is interaction-derived — it comes into existence through the act of the call itself. No pipeline was configured, no job scheduled. The relationship between source data, agent reasoning and output is created dynamically and may never recur in the same form. And the initiating entity is not a Job with a pre-declared name and run lifecycle; it is a runtime actor whose identity is known only at the moment it acts.
So what is missing is instrumentation at the MCP boundary, plus a modest extension of the lineage model to represent runtime actors. Neither is a rewrite — OpenLineage already has the entity model, the extensibility mechanism (facets, and the open `type` field from the jobless lineage work), and the consumer ecosystem.
And this is not an armchair observation. We have put exactly this question to both communities: an RFC in OpenLineage on lineage for runtime actor-initiated data interactions, and a corresponding tracking issue in the MCP repository. Both projects now sit under the Linux Foundation — OpenLineage in LF AI & Data, MCP in the Agentic AI Foundation — which gives the cross-community conversation a natural institutional home rather than each side solving it alone.
The asymmetry nobody has noticed
Here is the strangest part of the current landscape. The industry *is* connecting agents to lineage — in exactly one direction. MCP servers now exist that let agents query lineage graphs: ask an agent about upstream dependencies, root-cause a broken pipeline, explore the metadata estate conversationally.
The industry is building agents that can read the lineage graph. Nobody is making agents write to it.
Agents are becoming consumers of lineage while remaining invisible to it — the fastest-growing category of data pipeline on earth, and the only one producing no lineage events at all.
Three tracks, one question
Put the audit question to each track:
Track 1 answers with a shrug.
Track 2 answers: "the agent called the query_warehouse tool at 14:32." An access record. Not an answer — and no amount of agent telemetry, audit logging or governance documentation turns it into one.
Track 3 answers: these datasets were read, joined this way, landed in this prompt, in this session, shaped this output, consumed by these downstream systems — correlated to the exact trace of the agent behaviour that caused it. That is a GDPR answer. A BCBS 239 answer. A model risk answer.
Track 3 is the only track with an answer — and today, nobody is shipping it. But the conversation to change that has formally begun, in the open, in both communities.
So the obvious next question is: what would shipping it actually take? Firing OpenLineage at the MCP boundary, representing runtime actors in the lineage model, what belongs on the span versus the run event, where the cross-community RFC goes from here — and why open standards make all of it cheaper than it looks.
That is part three.
aigov!=aio11y
They are linked. But they are distinct. And only one track proves it.
About the author: Daniel Rolles is CEO and Founder of BearingNode, and one of the key authors of BearingNode's Data and Information Observability Framework. He has spent over 30 years in data, analytics and AI, and is the author of the OpenLineage RFC on lineage for runtime actor-initiated data interactions referenced in this series.
