i've been in enough post-mortems to recognize the move. the project didn't ship. the demo never made it past pilot. the rollout reached 8 active users out of 220. and the title of the meeting is always some variation of what went wrong with our ai project.
the model is the most visible thing in the system. that doesn't make it the cause.
the conversation goes through the same checkpoints every time. did we pick the right model? should we have used a different vendor? was the fine-tuning approach wrong? was our prompt strategy mature enough? these are real questions. they are almost never the answer.
by the time you trace any of these failures back far enough, the failure is somewhere boring. a copilot that drafted perfect answers while no one had decided who owned the final response. a sales assistant that summarized calls perfectly but depended on crm fields nobody filled in. a knowledge bot trained on six versions of the same policy. the model is what people watched. the failure was upstream. the model was asked to automate a reality the business had never made explicit.
- 01 "the model" the data underneath it couldn't answer a basic question consistently. the model was the messenger.
- 02 "change management" the actual workflow nobody touched. people were given access, not a reason.
- 03 "executive sponsorship" the sponsor was attached to the initiative, not accountable for the outcome. when the budget got tight, support evaporated.
- 04 "timing" nobody could say what had to be true before rollout. without a number, "ready" becomes a feeling.
- 05 "the wrong tool" the workflow it was supposed to fix was never coherent enough to fix with anything.
the misdiagnosis matters because it's expensive. it sends every retry in the wrong direction. another vendor evaluation. another rfp. another six-month integration. another meeting about ai strategy. meanwhile the actual problem — the workflow nobody mapped, the number nobody owned — is sitting where you left it, getting worse.
there's a quick test. when an ai initiative starts feeling stuck, ask: if the model in this pilot were perfect tomorrow, would the project succeed? if the answer is no — and most of the time it is — then the model was never the problem. you are debugging the wrong part of the stack.
if you're staring at an ai problem that won't move, stop tuning the model. go one layer down. then another. the real problem is usually boring, operational, and named something else. fix that — and the model finally gets to do the job you hired it for.
stop tuning the model. start auditing what's beneath it.