Why AI pilots stall at the process, not the model
Pilots of AI rarely fail because the model was too weak. They stall because the process around the model was never written down, has no owner, or runs half in a spreadsheet. Fix the process first and the same model that disappointed in the demo starts doing useful work.
Key takeaways
- A demo runs on a clean example. Production runs on the exceptions, and the exceptions are the process.
- The four usual gaps are ownership, data location, permissions and change history.
- Swapping the model does not close any of them.
- One workflow with an owner is a better first target than a company-wide assistant.
What the research says about failed AI initiatives
72% of organizations say process-related challenges have caused AI initiatives to fail
The respondents worked at companies with more than a thousand employees, which have process teams. Smaller firms rarely do, so I would expect the share there to be no lower.
Four gaps that stop a pilot
- Nobody owns the process
- The pilot team asks who decides how an exception is handled and gets three answers. An agent cannot be supervised by a committee that has not met.
- The data lives in two places
- Part of the record is in Jira or Monday.com and the rest is in a spreadsheet someone maintains by hand. The model sees half and answers confidently.
- Permissions are by convention
- People know not to open certain projects. An assistant with a service account does not know, and reads everything its token allows.
- Changes leave no trace
- When the output goes wrong, nobody can say whether the prompt, the model or last week’s workflow edit caused it.
Why a better model does not help
A stronger model reads the same half of the data and inherits the same permissions. It writes more fluent answers on top of the same gaps, which makes the errors harder to notice. Teams then conclude the technology is not ready, when the workflow was the part that was not ready.
The reverse also holds. With an owner, one source of data and a change history, a mid-range model is enough for most operational tasks.
Closing the gap before the next pilot
- Pick one workflow that somebody already measures.
- Name a person who can approve a change to it.
- Move the spreadsheet step into the system, or accept that the pilot covers only what the system holds.
- Check what a service account can read, and narrow it.
- Turn on whatever change history the platform offers.
The readiness checklist expands each step, and the scorecard turns them into a score in five minutes.
Where the process gap is not the problem
Some pilots do fail on the model: tasks that need exact arithmetic over long tables, or judgment a domain expert cannot explain either. If your process is owned, clean and logged and the output is still wrong, the task may not suit a language model yet, and saying so early is cheaper than a second pilot.
Frequently asked questions
How do I know whether my pilot stalled on process or on the model?
Do we need process mapping software first?
If a pilot stalled and nobody can say why, the audit finds out in two weeks.