aigov!=aio11y: We are making the same mistake with AI that we made with data
- Daniel Rolles
- Jul 5
- 5 min read
Somewhere in your organisation today, an AI agent queried a data warehouse, joined the results, reshaped them, and injected them into a prompt.
An ad-hoc data pipeline was created, executed and destroyed inside a chat session.
Your governance framework has a policy about it. Your telemetry captured that a tool was called, and how long it took. And nothing — anywhere — recorded what the data actually did.
If your organisation can't answer *"what data did this AI system access, and where did it go?"*, you don't have an AI governance problem. You have an AI observability problem wearing a governance badge.
A while ago I wrote that gov!=o11y: data governance and data observability are linked, but they are not the same thing. Governance is normative — it says what should happen. Observability is empirical — it shows what is happening.
The data-era version of the argument: observability overlaps governance, compliance, management and strategy — but is distinct from all four. Hold this picture in mind; we are about to redraw it for AI.
I am writing this because the industry is about to make the same mistake again. This time with AI, and this time at AI speed.
In shorthand: aigov!=aio11y
I was in those rooms
This isn't a history lesson from the outside. In 2013 I was sitting with newly appointed Chief Data Officers — predominantly in banks, predominantly in the shadow of the financial crisis and BCBS 239 — helping them work out what their brand-new mandate actually meant. The instruction from above was clear: get control of the data.
And I watched, from inside the room, what "control" became. Policies. Standards. Stewardship councils. Catalogues. Attestations. Governance frameworks with committees to oversee them and registers to record them.
We confused governance with observability, consistently, for the better part of a decade. We treated the existence of a controls framework as evidence that the thing being controlled was under control. It was not. A policy that says data must be classified tells you nothing about whether it is. A catalogue entry tells you what someone once believed about an asset, not what is true of it today.
I helped build some of those frameworks. That is exactly why I recognise what is being built around AI right now.
Meanwhile, something instructive was happening one floor down.
The fork in the road
While the data world was writing policy, the software and infrastructure world was instrumenting. SRE and DevOps culture treated visibility as an engineering discipline, not a documentation exercise. Telemetry became a first-class concern. OpenTelemetry emerged as a vendor-neutral standard, and logs, metrics and traces became the shared language of how systems evidence their own behaviour.
The infrastructure world moved on. The data, analytics and AI world got stuck on governance.
That fork in the road is why, a decade later, an engineer can trace a single request across forty microservices, while a CDO still struggles to answer "where does customer data actually live?" One community built an evidence layer. The other built a paper trail.
Data observability as a discipline only emerged seriously around 2019–2021 — roughly ten years behind its software equivalent. We are still paying down that gap.
It is happening again — faster
Now look at what is being built around AI.
AI governance frameworks. AI ethics boards. Acceptable use policies. Model risk committees. Principles documents. The EU AI Act compliance programme. All of it necessary. None of it sufficient. And all of it arriving — again — ahead of the instrumentation that would tell anyone whether any of it is actually working.
There is a twist this time, and it makes the conflation harder to spot, not easier. Some of the best-engineered AI "governance" stacks I have seen this year are, in fact, genuinely impressive observability stacks — wearing governance badges. The visibility is positioned as a feature of the governance: the reason to see things is so you can govern them. That framing should sound familiar. It is exactly how data observability tooling was sold as "data governance" in the 2010s, and it collapses two distinct disciplines into one.
The speedometer does not exist to serve the speed limit.
I'll return to one of those stacks in detail in part two — because what it captured brilliantly, and what it structurally could not capture, is the whole argument in miniature.
AI governance is normative. AI observability is empirical.
AI governance might say:
models must be risk-assessed before deployment
agents must only access authorised data sources
outputs must be monitored for bias and drift
human oversight must exist for consequential decisions
AI observability tells you:
what your models and agents are actually doing, right now, in production
which data sources an agent actually touched, and what it did with them
how behaviour is drifting as inputs, models and prompts change
whether the controls your governance framework assumes are actually firing
Without AI observability, AI governance is difficult to verify.
Without AI governance, AI observability lacks context for what should be measured.
Linked. But distinct.
Learn the lesson — but embrace the technology
Let me be clear about what this argument is *not*. It is not "slow down". It is not "AI is the new shadow IT, lock it down". The organisations that win this decade will be the ones that deploy agentic AI aggressively — *and* can evidence what it is doing. The lesson from data is not that governance was wrong; it is that governance without an evidence layer is write policy and hope.
Write policy and hope did not work for data, and data fails slowly — a lineage breaks, a quality score degrades over weeks. AI systems drift in hours. Agentic systems do not just produce outputs; they take actions. The gap between "what policy says should happen" and "what the system is actually doing" now has real-world consequences, not just analytical ones.
We spent ten years closing the gap for data. We do not have ten years this time. The good news: we do not need them — the standards, the patterns and the lessons already exist. We just have to actually apply them, at AI speed.
The question that decides everything
So here is where part one leaves you. A regulated organisation gets asked — by an auditor, a regulator, a data subject:
"What data did this AI system access, and where did it go?"
Every AI deployment on earth is currently on one of three tracks with respect to that question. Two of them cannot answer it. One of those two looks — convincingly, expensively — as if it can.
That is part two.
Same mistake. Harder problem.
aigov!=aio11y
They are linked. But they are distinct.
About the author: Daniel Rolles is CEO and Founder of BearingNode, and one of the key authors of BearingNode's Data and Information Observability Framework. He has spent over 30 years in data, analytics and AI, including the decade in which newly appointed Chief Data Officers in regulated industries turned governance mandates into real capability — the rooms this series is written from.
