We are doing our observability engineering in public
- Daniel Rolles
- 18 hours ago
- 5 min read
Governance is normative. It says what should happen.
Observability is empirical. It shows what is happening.
They are related, but they are not the same thing. The data industry spent much of the past decade proving it. Policies, councils, catalogues, attestations and stewardship frameworks were all treated as evidence that the thing being controlled was under control.
It was not.
A policy saying that data must be classified does not tell you whether it is.
We are now repeating the pattern with AI, only faster. AI governance frameworks, ethics boards, model risk committees, acceptable-use policies and EU AI Act programmes are necessary. But they are not sufficient on their own. Many are arriving ahead of the instrumentation needed to show whether the controls are working.
That gap is what BearingNode Lab is for. It is open today.
For regulated organisations, this is not an abstract distinction. A policy may describe the control. Only evidence can show whether the control operated when an AI system reached governed data.
What the lab is
BearingNode Lab is a source-available research repository where we do our observability and governance engineering in public.
It will contain reference implementations, specification proposals and contributions to the standards communities whose specifications we are asking to change.
The lab treats three domains as one problem rather than three:
AI governance, management and observability
Software and infrastructure governance, management and observability
Data and information governance, management and observability
In most enterprises, these are three teams, three tool budgets, three vocabularies and three sets of dashboards that rarely describe the same operating reality.
That separation was defensible for a long time. It is not any more.
The reason is narrow enough to state precisely: An AI agent reaching governed data creates two records at once.
One records what data was reached. The other records why it was reached and what followed. They are linked, but they are not the same kind of evidence. They have different owners, lifecycles and control obligations.
Those records should not be merged.
Telemetry is often sampled, short-lived and shaped for debugging. A governance record cannot be any of those things.
They need to connect at the right control point. Finding and standardising those points is the work.
The practical cost of the current separation is duplicated instrumentation, slower investigations and control evidence that cannot be assembled without manual reconciliation.
Why "observability" and not "monitoring"
Because the word is doing specific work.
Data and Information Observability (term of art D/I O11y) is:
"The body of knowledge and practices for monitoring the health, performance, and organisational impact of Data and Information assets, as well as the capabilities to steward those assets."
That is our definition from The Rise of Data and Information Observability, quoted verbatim.
The two clauses form one definition, not two alternatives: a body of knowledge and practices for monitoring, as well as the capabilities to steward.
Drop the second clause and you have a dashboard. Keep it and you have a discipline.
What is in it today: not very much, and that is the point
This first edition contains the lab's foundation only:
Licence
Contributor agreement
Governance
Security policy
Code of conduct
Provenance
Citation metadata
The principles we hold ourselves to
No workstream content. No code. Nothing to run.
We could have waited and launched with the engineering attached. We decided not to.
The terms on which the work will be released are load-bearing. They deserve to be read before anyone is looking at a demonstration.
If you want to understand what the licence permits, how we work with standards bodies, what we disclose about the way the lab was built or what happens to a pull request, it is all there now.
The first workstream is in preparation and will land as its own edition.
How the lab works
The lab publishes the principles it holds itself to. These three have cost us the most.
Primary sources, at a stated version
Specification behaviour is read from the specification.
Not from a summary, not from memory and not from a model's recollection.
We make arguments about what specifications do and do not carry. An argument built on a paraphrase is not an argument. This rule has already caught several of our own claims before anyone else had to.
The record ships, corrections included
Every workstream keeps a register of its risks, assumptions, issues and decisions.
We write it as we go and publish it without tidying away the inconvenient parts. Where we were wrong, the invalidation stays beside the original wording.
A register showing only the decisions that survived would be a marketing document, and everyone would know it.
Show the working
A number ships with the command that produced it.
An unreproduced result is marked unverified rather than presented as a finding. A script that prints a green tick and exits zero is a demonstration, not a test.
These are aims, not warranties. We will not always meet them.
Where we miss, the registers are where it should be recorded. That is precisely why they ship unpolished.
Source-available, and honest about why
The lab is published under the BearingNode Community and Research Licence.
It is source-available, not open source, and both halves of that description matter.
The thinking is intended to spread widely. Take the ideas, frameworks, patterns and architecture into your own work. Use them in open standards, open-source communities, research and teaching. Build independent implementations.
What requires our consent is the copying, redistribution or upstreaming of the repository's materials, and their commercial use.
For open-source and open-standards projects, the answer will usually be yes. Where it is, we will normally contribute the material directly under the project's own licence and process. That allows it to arrive cleanly licensed rather than encumbered.
One position is unusual enough to state openly.
We withhold consent for training commercial models on this work.
We explain the reason rather than simply prohibiting it. There is currently no established mechanism through which a rights-holder is compensated when their work is used to train a commercial model. There is no reliable per-use rail, collecting society or accounting mechanism.
Consent is the instrument available to us, so we are retaining it.
We are not standing outside this problem.
We built the lab using commercial and open-weight models. We paid for that access, and none of that money necessarily reached the people whose work trained them.
The lab's disclosure document sets this out in full, including the uncomfortable part.
Built with AI, and disclosed as such
This work is built with AI throughout, and the lab says so at length rather than in a footnote.
It explains what that means, what we do not claim as a result and why we ask readers to verify the work rather than adopt it uncritically.
If the work is to be held to the standards of the communities it addresses, those communities should know how it was made.
What we want from you
The lab has issue forms for exactly two things:
A bug
A question or challenge
The second is not decoration.
If you work in a regulated enterprise and believe the problem we are describing is not your problem, we want to hear the argument.
If you maintain one of the standards we are asking to change, we would rather be told that the proposal is wrong than be politely ignored.
If the code appears to follow a rule we have not written down, ask. The registers are where we should have said so.
BearingNode Lab: github.com/BearingNode/bearingnode-lab
Start with the README, then `PRINCIPLES.md`.
If you read only one thing, read what we say we do not claim.
