Your AI trusts the data catalogue more than your team ever did
For years, I have watched people quietly work around stale data catalogues. Descriptions written during a documentation push drift as the tables change, and the people using them compensate: they ask a colleague before trusting a table, check who else queries it, and learn which parts of the catalogue to ignore. The catalogue was never fully accurate. It did not need to be.
An AI agent will take every word at face value. The first wrong number it puts in a leadership report will look right: a customer count nobody questions until someone compares it with their own, by which time the figure has been discussed, and tracing it back takes days, because everyone checks the data first. The data will be fine. The agent will have read the wrong table, because the catalogue pointed it there.
This is the failure I expect most teams to meet in their first year of connecting agents to their data. A stale description is no longer a documentation gap. It is a production bug.
What people knew that the catalogue never held
An agent matches the question to the description that fits best. After a data platform migration, the old customer table is often kept "for a few weeks" and never removed. Its description still reads like the source of truth, while the new table's description is a single thin line. A person would ask which one is current, and would distrust anything called customers_v2_final. The agent picks the better-sounding description.
Meaning drifts quietly as well. Descriptions are written when a table is created and rarely revisited when its meaning changes: the definition of an active customer moves, and the column description still states the old rule.
Coverage is not accuracy
Most catalogues were rolled out as a one-off project. Schemas were ingested automatically, descriptions were filled in during a documentation sprint, and coverage was reported as the measure of success. The catalogues I have worked with looked complete by that measure, while many of their descriptions still described the system as it was during the sprint.
Nobody was assigned to keep them true afterwards, and replaced tables were never marked as deprecated. For an analyst, that was an inconvenience. Once an agent is connected, coverage looks excellent and accuracy is unknown, and the agent cannot tell the difference.
The tempting fix is to generate the missing and stale descriptions with AI. Descriptions generated from column names repeat the names, not the meaning, and they read as authoritative, which makes staleness harder to spot. AI can draft them, but someone who knows what the data means has to confirm them.
Nobody owns the sentence
Data engineers maintain schemas, not meaning. Business owners know the meaning but rarely open the catalogue. The platform team owns the tool, not what is written in it. Between them, nobody answers the only question that matters to an agent: is this description still true?
The leadership mandate
The CTO decides whether the data catalogue is documentation or production infrastructure, and the answer changes the moment an agent is connected to it. Research from MIT's Center for Information Systems Research makes the same point from the other side: the layer that carries business context needs a clearly designated owner, and organisations with more developed data curation practices were more than three times as likely to report effective data and AI initiatives. Naming an owner for each critical domain is the decision. The tooling follows.
The path forward
Before connecting an agent to the catalogue, measure accuracy, not coverage:
- List the 20 tables the AI use case will touch, and check whether each description is still true.
- Mark every replaced table as deprecated, so it can no longer win a match on its description.
- Make a description update part of the same change that alters a table's meaning.
- Name an owner for each critical domain, accountable for whether its descriptions are current.
The final check is a single question: would a new analyst get the right answer from the catalogue alone? That is exactly what the agent does every time it answers, without the Slack message that used to rescue the analyst.
