Narrow AI in operations: what works, what stalls
Companies launch AI pilots, but most never reach production. What distinguishes working cases from stalled ones.
Over the last several years I have seen dozens of conversations about "rolling out AI" - and significantly fewer cases where that led to something actually running in production. The gap between pilot and production remains wide, and it does not seem to be narrowing.
The reason, in my observation, is almost never the technology. It is how the task is defined and in what context it is launched.
Why pilots stall
First reason: the task is framed too broadly. "Predict customer churn" is not a task for a model - it is an entire workstream. There is no clear input, no clear output, no defined place in an operational process. You can train a model, but there is nowhere to apply it.
Second reason: the data is not ready. The pilot starts, data is pulled together at the last minute, it turns out that some of it is unsuitable and some is missing. Then either a long data preparation effort begins, or the pilot is declared "technically successful but not scalable".
Third reason: there is no user for the result. The model ran - now what? If there is no specific person or system that uses the output and changes their behaviour as a result, the value is zero.
What working cases have in common
I have seen several cases where AI genuinely stayed in operation. They shared a few characteristics.
Clear and narrow input. A document, an image, a set of numbers - something concrete, not "data from the system in general".
Defined and bounded output. Binary classification, a numeric score, a category from a closed list. Not "tell me about the customer", but "is this invoice normal or does it need review".
A defined place in the process. Who specifically sees the model's output? When? What do they do differently?
Willingness to accept errors. No model works perfectly. An organisation that is not prepared for the model to occasionally be wrong will stop any pilot at the first mistake.
How to think about the next pilot
Before launching something new, it is worth honestly answering a few questions:
- Can we describe the task as "input is X, output is Y" - concretely, without metaphor?
- Do we have the data needed for training and operation - right now, not "in theory"?
- Who specifically will use the model's output, how, and when?
- What will change in the operational process if the model works well?
- What will we do when the model makes a mistake - do we have a fallback?
These are not technical questions. They are operational. And they are the ones that most often determine whether a pilot reaches production or not.