Skip to main content

Defensibility Is Mostly Powered by Obscurity. AI Is Ending That.

I have a demo screen that does something people argue about.

Pick one blood draw, for one participant, at one visit in a clinical study. The screen shows everything the rules say should surround that single moment, in one place: the signed consent form, the training records of the people involved, who was authorized to do the work, their certificates, and the electronic-record controls the FDA requires (21 CFR Part 11). One visit, one person, and every piece of evidence that it was done correctly.

The reactions split into two camps. Some people say it's impossible. Others ask why anyone would want that level of detail and precision.

I've come to think the second reaction is the more honest one. It's the reaction of people who have learned that being precise is a risk. Most organizations defend themselves the same way: they keep the record thin.

I don't mean that as an accusation. I've heard people say it out loud, and once you hear it, you start seeing it everywhere.

Earlier this year I talked with a lawyer about how much detail a company should keep about how its decisions get made. To paraphrase his answer, no company wants records that could come back to bite it, and tech people often arrive with the idea of "wouldn't it be cool to have all this data" without asking whether there's a need for it. His point was that data you don't know the use for is an asset, but also a potentially massive liability. That's a big issue with big data and a bigger one with AI. It's why companies have retention policies, which are a company's rules for how long it keeps records before deleting them.

Around the same time, a quality leader at a health company told me, "We keep SOPs vague because otherwise sponsors and monitors will use them against us." An SOP is a standard operating procedure, the written instructions for how work is supposed to get done. Sponsors are the companies paying for the work, and monitors are the people they send to check on it. So the written rules stay fuzzy on purpose, because a precise rule is a rule you can be caught breaking.

Neither of these people is a villain. They're doing their jobs, and they've learned what the system rewards. If you can't be pinned down, you can't be blamed. A lot of us have quietly accepted that deal. Defensibility is mostly powered by obscurity.

And that deal has a price nobody puts on a budget line. The same vagueness that protects a company also makes every decision expensive to reconstruct, and when reconstruction is expensive, the whole industry is expensive. The honest answer to the lawyer's warning isn't to keep less. It's to keep the right things on purpose: the records that explain your actions and conclusions, and who signed off. Data you can't explain is a liability. Data that explains your decisions is what makes them defensible.

AI is breaking that deal.

Why AI changes the math

Obscurity worked because connecting the dots was expensive. Records live in dozens of systems. To rebuild the story of a single decision, someone had to pull records from five places, line them up, and read them. Almost nobody did that unless there was a lawsuit or a crisis.

AI does that kind of work all day, and it's cheap. Point it at your systems and it will stitch together a story about who decided what, when, and why.

And it isn't only your own people who will use it. A senior quality leader at a large biopharma site, fresh off a regulatory inspection, told me the inspectors arrived with questions that felt like his records had already been fed into an AI. About ten people sat in the room with the inspectors, and sixty or more worked in back rooms preparing for every angle the inspectors might take. He didn't describe them as adversaries. He described people trying to protect patients, who sometimes read the rules differently than his team does. That is the point. The people who come to look are capable and increasingly well equipped, and the same tools are heading to customers, sponsors, and anyone else with a reason to ask.

The trouble is that data rarely carries its history with it when it moves. A number gets exported from one tool, pasted into a spreadsheet, summarized in an email, and fed into a model. By the time it lands, nobody can tell where it came from, who approved it, or what it assumed. The AI can't tell either, so it often treats the number as a fact.

So two things happen at once. AI finds the places people were hiding, and it fills the gaps it can't see with confident guesses. The old defense turns into the new risk, because a thin record is exactly where AI starts guessing.

Transparency or defensibility?

A friend asked me recently which one this is really about. My answer was both, and one issue cuts both ways. Humans like places to hide, and AI is finding those places.

Everyone says they want transparency. It shows up in mission statements and conference talks, and a lot of the time it's there for show. The real question is whether any industry is ready for true transparency. I spend most of my time in healthcare and life sciences, but I don't think the pattern is unique to us. Anywhere decisions carry real consequences, you'll find people who learned that the safest record is the one that doesn't say much.

I don't think the answer is to rip the covers off and force everything into the open. That mostly teaches people to hide better.

The decision in front of you

Here is why this matters now, and not in five years.

Every organization is about to give AI access to its information. When that happens, it has to make an architectural decision, whether it admits that or not. One option is to keep patching: more connectors between systems, more people stitching answers together by hand, more spreadsheets with scripts that nobody remembers writing. In my experience, most of the money on a big project already goes to this kind of work, connecting and reshaping data, before anything new gets built. An AI that sits on top of that setup inherits the same gaps and fills them with guesses.

The other option is to treat what the organization knows as a shared asset, a kind of institutional brain. It holds the data, what the data means, how people actually run the work, and the adjustments they've made along the way. The knowledge doesn't walk out the door when someone retires. A new person, or an AI, can pick up where the last one left off.

That brain is only as good as what you can trust inside it. A memory that can't show where its facts came from is a very confident rumor. That's where evidence comes in.

What evidence adds

Organizations are about to build a lot of AI applications, fast. Many will be short-lived. Someone builds one for a single job, uses it for a few weeks, then throws it away. That's fine, and often it's good. But those apps need a framework that lets people build them in a defensible way. Otherwise every quick app becomes one more place where an unchecked assumption turns into a "fact."

That framework is what we are building. We call it evidence infrastructure. The goal is that humans don't have to dig for answers and AI doesn't have to guess.

Most software already keeps some kind of audit trail, a log of who changed what and when, inside its own walls. Evidence infrastructure keeps the evidence attached to a piece of information as it moves between systems. For any number, document, or decision, you can see who created it, what created it, why, when, where, and with what model if AI was involved. You can also see whether anyone checked it. When an AI picks it up, it can tell whether the thing was verified or whether it's somebody's estimate from three months ago.

A shared memory also needs to handle change honestly. When a decision gets reversed, for example a claim denied three times and then approved after a ruling, the old denials shouldn't be erased. They should be marked as replaced. Otherwise an AI reading the whole file will cheerfully report that the claim was denied.

That changes what "defensible" means. Today it often means there isn't enough on record to prove you were wrong. With real evidence, it means you can show what you knew, why you decided, and who signed off. And that job is yours. If a regulator, a customer, or a court asks how a decision got made, the burden of proof is on you, and pointing at a software vendor doesn't meet it. The first kind of defensibility falls apart the moment someone connects the dots. The second kind holds up better the more people look.

People still make the calls. The evidence keeps track of why.

The guardrail

None of this means exposing everything to everyone.

Some walls exist on purpose. Patient privacy is one. So is need-to-know access, where people see only what their role requires, along with HR files and parts of finance. Connecting systems must not tear those walls down. Other walls are only friction, like systems that were never built to talk to each other or a process nobody got around to fixing.

Anything that connects your systems has to know the difference. It should keep the walls that protect people and remove the friction. If it can't tell them apart, it will either leak what should stay private or leave the hiding places standing and call them privacy.

So, are you ready?

Try this test. If an AI connected every system in your organization tomorrow, and respected the walls that are supposed to be there, what would it find? Decisions backed by evidence, or vague procedures and retention policies doing the real work?

If it's the second one, AI will find it sooner or later. Better to decide now how your organization will remember, and build the evidence in on your own terms, so that when someone connects the dots, the picture holds up.

Obscurity was a decent defense when nobody could afford to look. Now everyone can.

Subscribe via RSS