Why AI projects die before they produce results: five recurring patterns

2018 was a busy year for AI: BERT raised the bar for NLP, the number of companies that launched AI pilots grew sharply, and venture funding into AI keeps rising. Against that backdrop there is a less visible statistic: most AI projects do not reach production use.

I see this often enough to recognise recurring patterns. Not a new unique situation each time - the same few mistakes in different variations.

Pattern 1: technology looking for a problem

The most common variant - start with the technology rather than the problem. "We want to introduce machine learning into our process." Which process? What problem does it solve? What is the cost of that problem?

When there are no answers, the team starts looking for a task that would fit the technology. This is the wrong order. The problem must come before the technology.

Pattern 2: unrealistic pilot expectations

A pilot is launched with the expectation that "if results are good, we scale immediately". But the criteria for "good results" were not defined upfront. Technical metrics look reasonable, but the link to business outcomes was never established.

After a few months the team presents a "working model" that is technically sound but whose impact on business metrics cannot be measured. The project gets stuck in ambiguity.

Pattern 3: data underestimated

This is discussed a lot, but the pattern does not disappear. A pilot runs on "ready" historical data. When the team tries to move to production, it turns out that getting this data in real time is difficult, it is not in the right format, or it does not exist at all - only something similar but not quite the same.

The data work turns out to be larger than the model work. This is a normal situation, but it needs to be in the plan.

Pattern 4: no owner after launch

The data science team builds the pilot. They deploy the system to production. Then they move to the next project. The system runs unsupervised. A few months later quality has degraded, the input data has changed, nobody noticed.

An AI system is not ordinary functionality that you write once and forget. It is a living system that needs regular attention.

Pattern 5: internal resistance not accounted for

A system classifies customer requests - which means someone on the operations team is losing part of their job or changing the nature of their work. A system recommends credit decisions - which means credit analysts see it as a threat to their expertise.

If this resistance is not addressed explicitly, the system works technically but is not used. Or it is used formally, without real trust.

What to do about it

These patterns repeat because they are not technical - they are managerial.

A few questions worth asking before the start:

1. What specific business problem are we solving, and how will we measure success in business terms?
2. Where will the data come from in production - has someone verified this?
3. Who will own the system after launch, and what does that ownership include?
4. How will the work of people whose activities the system affects change, and how are we handling that?
5. What is the minimum pilot result we would accept - defined before we start?

AI projects are not inherently doomed. They fail when the management preparation does not match the technical ambition.