Why AI projects fail, and how to actually ship one that sticks
AI projects do not fail because the models are not good enough. They fail because teams treat shipping AI as a demo problem when it is a reliability problem. The projects that stick measure accuracy on real data before they trust it, check the model’s output with deterministic rules, keep a human on the exceptions, and scope one narrow workflow with a baseline and a kill switch. The failure rate is real, more than 80% by RAND’s read, roughly twice the rate of IT projects without AI. But it is usually not the model that fails. It is the missing engineering around it.
Why do most AI projects fail?
Most AI projects fail before the model is ever the problem. The biggest single cause is that no one defined the job precisely enough to build for, or to know when it is working. RAND interviewed 65 experienced AI practitioners and engineers for its 2024 report on the root causes of AI project failure. Of the five root causes it found, only one is technical, and it is real, sometimes a project genuinely asks the models for something still beyond them. But that is one cause of five, and not the common one. The first and largest is that leaders misunderstand, or miscommunicate, the problem the project is meant to solve.
Worse, the trend is heading the wrong way. S&P Global’s 451 Research surveyed 1,006 organisations and found the share of companies abandoning most of their AI initiatives rose from 17% to 42% in a single year, with the average organisation scrapping 46% of its proof-of-concepts before production. Gartner said in June 2025 that it expects more than 40% of agentic AI projects to be cancelled by the end of 2027.
Read that trend against the technology. Abandonment rose sharply through the same period in which the models improved fastest. That does not prove the models are blameless, a boom in adoption also means more projects to walk away from. But if raw capability were the binding constraint, better models should have pulled the failure rate down, and instead it climbed. It is a strong signal for the argument of this piece, not a proof of it.
One caution before any of these numbers settle in. The headline statistics measure different failures, and they are routinely quoted as if they measured the same one.
| The figure | What it actually measures |
|---|---|
| More than 80% of AI projects fail (RAND, 2024) | A figure RAND itself frames as “by some estimates”, drawn from the literature around its 65 practitioner interviews |
| 95% of pilots show no P&L impact (MIT NANDA, 2025) | No measurable profit-and-loss impact within a short observation window (the caveats are in the next section) |
| 42% (S&P Global / 451 Research, 2025) | The share of companies abandoning most of their AI initiatives, up from 17% a year earlier |
| 46% (S&P Global / 451 Research, 2025) | The average share of proof-of-concept projects scrapped before production |
| 88% (IDC and Lenovo, 2025) | The share of proof-of-concepts that never reach widescale deployment |
| More than 40% (Gartner, 2025) | A forecast, the share of agentic AI projects Gartner expects to be cancelled by the end of 2027 |
None of these is the same failure. When this piece says a project failed, it means the project never became a production system the business trusts and keeps using.
If the models are this good now, why do the projects still fail?
The model is the easy part. Everyone runs the same frontier models now, so a working demo is cheap and proves very little. The hard part, and the part most projects skip, is making the output reliable enough to trust with real work, on your real data, at real volume, without a person checking every answer.
MIT’s 2025 GenAI Divide report is best known for one number, 95% of enterprise pilots showing no measurable P&L impact. Its more useful sentence is the mechanism, in the report’s own words. The divide “does not seem to be driven by model quality or regulation, but seems to be determined by approach.”
That report deserves its caveats, so here they are. The sample is small and inconsistently reported, “no P&L impact” is a narrow success test over a short window, the report is not peer reviewed, and its conclusion promotes NANDA, an MIT Media Lab project selling agentic AI membership at a reported $250,000. We cite it anyway because even its critics concede the core finding. The divide is in the approach, not in the models.
McKinsey’s 2025 State of AI survey of 1,993 organisations shows the same gap from the other side. 88% of organisations now use AI in at least one function, only 39% report any impact on earnings, and about 6% qualify as high performers. Access to capability is near universal and value is rare, so the bottleneck is not the capability. Fortune’s follow-up reporting on the MIT study added a telling detail. Executives tended to blame model capability, and they were wrong.
One honest complication. Employees at the large majority of firms get real value from personal AI tools while the official pilots stall. That is not a counterexample, it is the boundary of the claim. Personal use works because the stakes are low and a human checks every output before it matters. An unattended workflow running the same job thousands of times a month is a different reliability class. That class is what the rest of this piece is about.
What do the teams that actually ship do differently?
The teams that ship treat AI as an engineering problem and produce engineering artefacts. An evaluation suite, an exception queue, a measured baseline. They do not run it as a transformation programme. They start from a business pain that already costs money, and they redesign the process around what the model can and cannot do, instead of bolting AI onto the process they already had.
McKinsey’s survey found its high performers were roughly three times more likely to have fundamentally redesigned their workflows around AI. McKinsey itself notes that one year of survey data cannot separate cause from correlation, and we quote that caveat deliberately. It is the honest reading, and the direction matches what we see in delivery. MIT’s report found the same shape, the successful minority embedded AI deep into high-value workflows, with memory and feedback loops, rather than scattering pilots across departments.
The strongest counterargument to a firm like ours sits in the same MIT data. Purchased AI solutions succeeded roughly 67% of the time, against roughly 33% for internal builds. Taken at face value, that says buy, never build. Look at what was measured, though. The study compared enterprise internal teams against bought products. A senior external engineering team building into your own stack is a third category the study never measured. It pairs the ownership of a build with the shipping discipline of a vendor, the half each measured category was missing.
How do you make an AI workflow reliable enough to trust with real work?
Reliability starts with measurement. You measure the system on your own real documents before you trust it, and you put deterministic checks over the model’s guesses. The model extracts, code verifies. An answer that breaks your rules is caught by logic you can inspect, not posted to your systems.
Start with error analysis, not metrics. Hamel Husain, whose writing on evals has become practitioner canon, puts the discipline plainly. Read the real traces, categorise the failures, and build your test sets from production data. If a model is judging another model’s output, validate the judge against human labels before trusting its scores. The anti-pattern is evaluating on vibes, with vague judge questions like “is this helpful?” standing in for measurement.
Accountants solved the second half of this problem decades before LLMs existed. The three-way match, purchase order against goods receipt against invoice, is a deterministic control formal enough to live in audit frameworks like COSO and SOX. The same pattern applies directly to AI extraction. Force the output into a schema, then check it with rules that do not guess. Line items must sum to the total, dates must parse, the supplier must exist in the master record. The model proposes, the code disposes.
The trap is the data the demo ran on. Demo documents are clean. Production documents carry encoding failures, missing fields, and formats nobody warned you about. An evaluation built on synthetic samples measures the wrong distribution, so the evaluation set must be built from your real documents, worst cases included. This method, eval-driven development with deterministic checks over the model, is the backbone of our AI Workflows service.
Where does the human stay in the loop?
A human stays on the exceptions, not on everything. Every extracted field carries a confidence score. Clear cases pass straight through, only the uncertain ones reach a person, and that person’s corrections become new test cases. A human who reviews everything is not oversight, it is a bottleneck that degrades into rubber-stamping.
The architecture is standard, and the working numbers below are practitioner rules of thumb, starting points you tune against your own data, not fixed constants or cited findings. Start around 80 to 85% confidence for document extraction, then tune in production, sized to the review capacity your team actually has. The calibration signal is the review-to-correction rate. If reviewers change roughly 5 to 10% of what they see, the threshold is earning its keep. If nearly everything they see passes unchanged, the gate is too conservative and it is wasting people.
The failure mode is easy to picture. Set the gate too high, say route everything under 90% confidence to a person, and a large share of the work floods the review queue in the first week, and overwhelmed reviewers start approving without reading. The oversight is then technically running and practically absent.
Route fields, not documents. Microsoft’s engineering guidance for its own document AI recommends posting high-confidence fields downstream and sending only the uncertain fields to a person. Zapier’s survey of more than 500 enterprise leaders found human-in-the-loop the most common approach to managing AI agents heading into 2026, at 38%. The craft is in the exception queue, which deserves the same design rigour as the happy path, because the exceptions are where the real work lives. The two governed agents on our work page run this architecture in production, a support agent that resolves 60% of tickets end to end and escalates the rest, and a front desk assistant that resolves 70% of enquiries and hands anything clinical to a person.
How do you scope a first AI project so it ships in weeks, not never?
Scope one narrow workflow that already costs real money, measure its baseline, and agree the kill criteria before you build. Run the cheaper test first. If a plain rule or a single model call solves the problem, it never needed to be an AI agent at all, and finding that out early is a win, not a failure.
The common 2025 failure has a name, pilot paralysis. Proof-of-concepts launched in safe sandboxes with no designed path to production, while integration, authentication and compliance wait unaddressed until the go-live date. The discipline that prevents it is unglamorous. Hold an internal build to the same phase gates a vendor rollout would get.
A measured baseline is what makes the phase gates real. APQC’s invoice benchmark is the worked example. Processing a single invoice costs top performers $1.77 and bottom performers $10.89, on 2023 data. That five-fold spread is the head-room argument, and your own number is the baseline. If you do not know your cost per document, cycle time, or error rate before the build, you cannot know whether the build worked.
On timing, we say weeks because that is what a properly scoped first workflow takes in our own delivery, and we label that as our claim, not an industry statistic. We looked for a credible primary figure on time-to-ship for a first AI workflow and did not find one. The precise-sounding numbers in circulation trace back to reports that do not exist.
How do you tell early that an AI project is heading for failure?
The earliest warning sign is that no one can state the problem, or the number that means success, in one sentence. Close behind, the demo has never touched production data, and the integrations are still mocked. Any of these at kick-off predicts the outcome better than any benchmark of the models.
The first sign is RAND’s number one root cause. Leaders misunderstand or miscommunicate the problem AI is meant to solve, and in some cases understand AI only as a buzzword, without realising a simpler and cheaper solution exists. The second and third signs are the demo’s hidden debts. A pilot on curated data masks latency under load, hallucination on real inputs, and the guardrails nobody built, and a mocked integration hides the authentication, rate limits and partial failures that are most of the production work.
The quietest sign is the most decisive. A project with no evaluation running cannot even know it is failing. The dashboard says shipped, and nobody can say whether it works.
The remedy is decided at kick-off, not discovered at the post-mortem. Set the kill criteria before the build, with the measured baseline to judge them against, and treat the kill switch as a feature rather than a defeat. We found no primary research quantifying how many teams pre-commit kill criteria, so take this as our method, not an industry finding. A project that can be stopped cheaply at a known point is a project a sensible business can start.
FAQ
Why do most AI projects fail? Most fail before the model matters, because no one defined the problem precisely enough to build for or measure, and of RAND’s five root causes only one is technical.
If the models are this good, why do projects still fail? The models are near universal, so a demo is cheap, and the engineering that makes output reliable on real data is the part most projects skip.
What do the teams that ship do differently? They treat AI as an engineering problem with evaluations, exception queues and a measured baseline, start from a pain that already costs money, and build into their own stack with vendor-grade shipping discipline.
How do you make an AI workflow reliable? Measure it on your own real documents before trusting it, and layer deterministic checks over the model so that code verifies what the model extracts.
Where does the human stay in the loop? On the exceptions, with confidence thresholds routing only uncertain cases to a person, whose corrections become new test cases.
How do you scope a first AI project? One narrow workflow that already costs real money, a measured baseline, kill criteria agreed before the build, and the cheaper non-AI test run first.
How do you tell early that a project is failing? No one can state the problem or the success number, the demo has never touched production data, the integrations are mocked, and no evaluation is running.
Written by Stavros Symeonidis, founding engineer at otum.