the symptom is always the same. the team picks a use case. they wire up a model. the model gives plausible-sounding answers that turn out to be wrong in places nobody can audit. then the conversation turns to model selection, prompt engineering, retrieval strategy — anywhere except the actual cause. the model isn't the problem. it's the only thing in the room that can be debugged.
the model can't reason about a business that hasn't written itself down.
the actual cause is that the model is being asked to reason about a business that exists in fragments. a definition that means three different things in three different departments. a policy that's a deck from 2023 and a slack thread from last week. a decision made in a meeting nobody recorded. the model isn't hallucinating. it's interpolating — filling in the parts no one wrote down, with whatever it found nearby.
the work of being ai-ready isn't a tech project. it's a knowledge project. and not knowledge in the wikipedia sense — knowledge in the make this company explicit to itself sense. it doesn't require a data scientist. it requires the same patient documentation discipline that turns a startup into an institution: writing down what's been true, what's been decided, what's been agreed, in a place where someone — or something — can find it without having to ask.
- 01 definitions the words your company uses that mean specific things here, written down once. when two teams use "customer" differently, the model has to pick one. you don't want to find out which.
- 02 decisions not just what was decided — why. "we discount above 200 units" is incomplete. "we discount above 200 units because last year's churn analysis showed the price-sensitive segment lives there" survives a model trying to apply the rule in a different context.
- 03 policies the current version, dated, with the previous version preserved. policy that lives in three decks is policy that doesn't exist.
- 04 context the unwritten assumptions made explicit. "we only quote in eur unless the client is uk-based" is the kind of rule that lives in seven people's heads and breaks every model trained without it.
- 05 ownership every knowledge block has a name attached. when a definition changes, someone updates it. when no one updates it, the model trusts something stale until someone notices.
this is the most expensive and least visible work in any ai initiative. it doesn't show up in a demo. it has no benchmark. it has no obvious before-and-after metric. it's done by the people who already know the answers — which means it's done at the cost of whatever else they were doing. it takes months. it produces no slides. and the company that does it doesn't look like it's working on ai at all. it looks like it's writing a lot of things down. at ai compass, the first month of any serious engagement is exactly this — interviewing the people who carry the unwritten rules, and turning what they say into something a model can read.
there are two reasons most companies skip it. the first is that knowledge work is harder to measure than tech work — there's no model card for "we have a single canonical definition of customer." the second is that knowledge work is political. canonicalizing a definition means deciding which team's version wins. that conversation has been deferred since 2017. it doesn't get easier when ai shows up — but ai is the first thing that forces it.
the move is to stop asking "are we ready for ai?" and start asking "can the company explain itself without a human interpreter?" if the answer is no, the work isn't ai. the work is making the company legible. once that's done, the model becomes a much smaller project than it looked at the start.
the model is only as good as the company that wrote itself down for it.