Data Classification at Scale: How to Actually Find and Tag Sensitive Data

Every access-control and AI-governance argument I've made rests on a layer nobody wants to talk about. Attribute-based access control only works if columns are tagged accurately. Agent autonomy tiers only hold if the sensitivity labels they map onto are current. Data contracts only keep PII protected if something declared it PII in the first place. All of that is downstream of one prerequisite — data classification — and it's the piece most programs assume already exists, gesture at, and never actually build. This is the post about building it.

Classification Is the Load-Bearing Layer

I keep landing on the same sentence from different directions: an access model is only as trustworthy as the classification underneath it. In the piece on moving from Unity Catalog grants to ABAC, the whole payoff of attaching policy to a PII tag instead of to individual tables collapses the moment the tags are wrong — you don't get better governance, you get a policy engine confidently asserting that mislabeled data is correctly governed. In the piece on governing AI-agent data access, the tier table that keeps a low-autonomy agent away from regulated data is only as real as the sensitivity column it reads. Both posts treat classification as prerequisite infrastructure and then move on, because the access mechanics are the interesting part. But the reason so many governance programs stall isn't the policy engine. It's that the classification the engine needs was never built to a standard you could enforce against.

The uncomfortable part is that classification is unglamorous, continuous work, and the payoff shows up as an absence — incidents that don't happen, access nobody accidentally granted. That makes it perennially easy to defer in favor of the shiny policy feature it's supposed to support. The result is a large fraction of enterprise "governed" estates where the map stopped matching the territory years ago and no one has looked.

Agree on the Taxonomy Before You Tag Anything

The first failure happens before a single scan runs: teams start tagging without agreeing on what the tags mean. One team's restricted is another team's internal, "confidential" and "sensitive" get used interchangeably, and six months later you have a classification vocabulary with no shared definition, which is worse than no classification at all because it looks finished.

Keep the taxonomy small enough that a non-specialist can apply it consistently. Four tiers cover almost every real need, and each one needs a one-sentence test a reviewer can actually run:

The point of the one-sentence test is that classification decisions get made hundreds of times by people who are not governance specialists. If a tier requires a policy expert to adjudicate every case, it won't be applied consistently, and inconsistency is the specific failure that makes the whole layer untrustworthy. Resist the urge to add a fifth and sixth tier for edge cases; push the edge cases into the nearest existing tier and let the access policy handle the nuance downstream.

Discovery: Three Signals, Not One

Once the taxonomy is fixed, the actual work is finding where the sensitive data lives — and at any real scale you cannot do it by hand. The mistake is trusting a single signal. Each of the three cheap signals is individually wrong often enough to be dangerous alone, and right often enough to be useful together.

Metadata and column-name heuristics. The cheapest pass: a column literally named email, ssn, or dob is a strong prior. This catches the obvious cases for almost no cost, and it is completely blind to anything mislabeled — the notes field somebody's been pasting national IDs into, the attr_7 column that holds phone numbers, the free-text blob that contains everything. Name-only classification produces a clean-looking report and a false sense of coverage.

Value-level detectors. The pass that earns its keep: sample the actual data and run pattern matchers and dictionary lookups against the values. Structured identifiers — email addresses, card numbers, national IDs — have recognizable formats that a regex plus a checksum can flag with high precision. Dictionaries catch the categorical cases: a column whose values are mostly known first names, or valid city names, or ICD codes. This is what finds the sensitive data the schema never advertised. The cost is that you have to read data, which means sampling strategy matters: a small random sample per column is usually enough to classify it, and you rarely need to scan every row to know a column holds emails.

Human confirmation. Neither machine signal knows meaning. A detector can tell you a column is full of nine-digit numbers; it cannot tell you whether they're national IDs, order numbers, or device identifiers — and the sensitivity depends entirely on which. So the machine's job is to propose, ranked by confidence, and a steward's job is to confirm or reject. High-confidence, high-agreement cases (name says email, values match the email pattern) can auto-apply; the ambiguous middle is exactly where you route a human, because that's where automation is most likely to be confidently wrong.

Tune for Recall, Because the Errors Aren't Symmetric

The single most important tuning decision in a classification pipeline is which direction to be wrong in, and it isn't a coin flip. A missed regulated column — PII that never got tagged — is a compliance exposure that surfaces during a breach or an audit, at the worst possible time. A false positive — an order-number column flagged as sensitive — costs a reviewer thirty seconds to reject. Those two errors are nowhere near equal, so the pipeline should be tuned to favor recall: over-flag, and make rejection cheap and fast, rather than optimizing for a tidy precision number that quietly lets regulated columns slip through unlabeled.

This is the same asymmetry that shows up in adversarial verification of AI output, where the verifier is deliberately biased toward "refuted" because a false pass is far more expensive than a false flag. Classification is the governance version of the same principle: the reviewer's time is the cheap resource, and the missed sensitive column is the expensive failure, so you spend the former to eliminate the latter.

The Part Everyone Skips: Keeping It Current

A classification effort run as a one-time project has a half-life measured in weeks. New tables land. A column gets repurposed. A schema migration moves a PII field into a table nobody re-scanned. Every one of those silently invalidates the classification, and every access policy and agent tier built on top inherits the staleness without any signal that it happened. I directed an enterprise catalog rollout at a Fortune-200 agriscience company specifically because keeping classification current across cross-functional teams was the hard part — not the initial scan, but the fact that the estate never stops changing and the labels have to change with it.

The durable fix is to stop running classification on a calendar and start running it on change. Re-scan when a table's schema changes or new columns appear, rather than waiting for an annual review to discover six months of drift. Alert when unclassified data shows up in a governed zone, so the default state of "no tag" becomes visible instead of silent. And — the connection that makes this stick — bind the sensitivity tag to the column through a data contract, so that a producer can't quietly widen what a field contains, or drop its classification, without the change surfacing at the boundary. Contracts are how classification stays honest at the point of production; without that link, tags are a snapshot that ages the moment you take it.

Build or Buy — Mechanism Over Product

You do not need a six-figure platform to start. Discovery is pattern matching plus dictionary lookups plus metadata heuristics; the tag store can be your catalog's native tags, a governed metadata table, or a column property in your warehouse. What makes classification real is that it's owned, reviewed on change, and enforced downstream — not the logo on the scanner. Commercial catalogs earn their cost at scale by adding workflow, lineage, pre-built regulated-data detectors, and the review queues that turn "we ran a scan once" into an operating process; I walk through that buy-vs-build math in the post on governance tool RFPs and total cost of ownership. But the sequencing advice is the same one I give for contracts: start with the enforcement point you already control, prove the process works on the two or three datasets that have already burned you, and let demonstrated value — not a platform purchase — drive the expansion.

The definitions feeding all of this have to be governed too. If the same terms mean different things to different teams, your classification inherits that ambiguity directly — which is the argument in the data-dictionary-as-governance piece, and it applies to sensitivity labels exactly as much as to business metrics. Classification and definition are two halves of the same discipline: one says what a field means, the other says how sensitive it is, and neither is trustworthy if the other is loose.