Your Data Dictionary Is Your RAG Governance Layer

Every team standing up retrieval-augmented generation right now is rediscovering, the hard way, why data dictionaries exist. I've spent a decade on the boring side of enterprise data — definitions, taxonomy, stewardship — and I've spent the last two years building RAG pipelines by hand on my own hardware. Watching those two worlds collide has been the most useful thing that's happened to my governance work in years, because it turns out the collision only goes one direction: the dictionary was always retrieval infrastructure. Nobody just needed it yet.

What Breaks When RAG Retrieves From Ungoverned Sources

A RAG system doesn't know a definition is stale. It doesn't know two departments use the same word to mean different things. It doesn't know which document in the index is the authoritative one and which is a draft somebody forgot to archive. It retrieves whatever is closest in vector space and hands it to the model with total confidence. Three failure modes show up almost immediately once you point retrieval at a real, ungoverned data estate.

Stale definitions retrieved as truth. Index a shared drive or a wiki and you index every superseded version along with the current one. The model has no signal that a policy doc from three reorganizations ago is dead. It reads confidently, sounds authoritative, and answers with content nobody should be citing anymore. The retrieval layer doesn't fail loudly here — it fails quietly, which is worse.

Homonym collisions poisoning answers. This is the one that breaks people's assumptions fastest. "Margin" means something specific in finance — the gap between revenue and cost of goods sold. It means something else entirely in sales — the discount room left in a deal. Both usages show up in the same document corpus, sometimes in the same meeting transcript. A RAG system with no classification layer treats them as the same token and blends the context. Ask a finance question and you can get a sales-flavored answer stitched in, wrong in a way that's hard to spot because it reads fluently.

No lineage means no way to trace a wrong answer. When a RAG system gives someone a bad number, the first question is always "where did that come from?" Without lineage — without a classification and ownership trail back to a system of record — you're grepping through embeddings trying to reconstruct which chunk, from which document, ingested on which date, produced the hallucination-adjacent answer. That's not a debugging process, that's archaeology, and it happens after someone has already acted on the wrong answer.

The Dictionary Work Already Solved This

Here's what surprised me. In a governance engagement I led some years back, the mandate was straightforward on paper: build the foundational data policies, produce a comprehensive data dictionary, standardize definitions and taxonomy across departments that had never agreed on shared vocabulary, and stand up stewardship practices so the dictionary didn't rot the moment the project ended. None of that was built with AI in mind. It was built because finance and operations and sales all had their own private meaning for the same fifteen core terms, and nobody could reconcile a report across departments without a translation layer sitting between them.

That translation layer maps almost exactly onto what RAG retrieval needs, term for term:

The workshops that used to feel like the slow, unglamorous part of a governance rollout — getting finance and sales in a room to agree on what "margin" means in each of their contexts, documenting both, and tagging which one applies where — are the exact workshops that make a RAG system answer correctly instead of confidently. Cross-departmental terminology conflicts were never just a communication problem. They were an unaddressed retrieval-ambiguity problem, decades before retrieval was the thing doing the reading.

The Bridge: Practical Steps

If you already have dictionary and taxonomy work sitting in a catalog somewhere, or you're starting from close to nothing, the bridge into RAG governance is three concrete moves, roughly in priority order.

• Embed the dictionary itself as a retrieval source. Don't treat the dictionary as documentation that lives outside the pipeline. Chunk and embed it like any other corpus, and weight it so a definition entry outranks an incidental mention of the same term buried in a meeting transcript. When "margin" comes up, the system should retrieve the definition first and the context-specific usage second, not the other way around.
• Tag chunks with catalog classifications. Every chunk that goes into the index should carry the same classification metadata your catalog already assigns at the table or column level — domain, sensitivity, department of origin. That metadata is what lets a retrieval filter say "only pull finance-classified content for this query" instead of blending domains silently.
• Gate retrieval by sensitivity tier. The same tiering that already governs who can query a restricted table should govern what a RAG system is allowed to retrieve on someone's behalf. A system answering a question for a general audience shouldn't have retrieval access to content classified for a restricted tier, even if the semantic match is strong. Fail closed, not open.

None of these three steps require new tooling categories. They require applying tooling you likely already have — catalog classifications, stewardship ownership records, sensitivity tiers — to a retrieval pipeline that was built without them in mind.

Where the Engineering Meets the Governance

I don't think this thesis holds if you've only done one side of it. Governance people who haven't built a RAG pipeline tend to over-index on process and under-estimate how much of this can be enforced mechanically, in code, at ingestion time, rather than through another committee. Engineers who haven't done dictionary work tend to underestimate how much of "the model gave a wrong answer" traces back to an unresolved definition conflict that predates the model by a decade.

Running my own local inference fleet — the same setup I've written about in the multi-machine AI fleet post — forced me to build retrieval pipelines from the ingestion layer up, which means I've had to solve the stale-definition and homonym problems in code, not just in policy. It's the same problem twice: once with a facilitation workshop and a shared spreadsheet, once with a tagging pipeline and a vector index. The engineering side just makes the cost of skipping governance visible faster, because a bad answer shows up in a chat window in seconds instead of a quarterly report in a month.

This also isn't just a RAG problem — it's the same access question that comes up once agents start acting on retrieved data rather than just answering questions with it, which I get into in the piece on governing AI agent data access. And the ownership half of the mapping above — who corrects a bad definition, who gets pinged when a classification goes stale — only works if stewardship is a running practice and not a one-time deliverable, which is the whole subject of what stewardship rollout looks like three months in, after the initial workshop energy has worn off.

The Thesis, Plainly

The boring governance artifacts — the dictionary nobody reads until they need it, the taxonomy that took eight workshops to agree on, the stewardship roster that assigns an actual human to an actual term — are suddenly load-bearing AI infrastructure. Not because anyone redesigned them for that purpose, but because RAG retrieval turned out to need precisely the three things dictionary work has always produced: authoritative definitions, classification-based filtering, and an ownership trail. Companies bolting RAG onto their data estate without that foundation aren't skipping a step. They're rebuilding it badly, chunk by chunk, one hallucinated answer at a time, and calling it AI risk when it's really an old governance gap wearing a new interface.