Why Some AI Projects Transform Businesses, And Others Stall

Enterprise AI isn't defined by the technology you choose. It's defined by everything you build around it.

Artificial intelligence has reached an interesting point in its evolution. 


Just a few years ago, the conversation was whether businesses should invest in AI at all. Today, that question has largely been answered. According to Stanford’s 2026 AI Index, 88% of surveyed organizations now use AI in at least one business function, while generative AI adoption continues to accelerate across industries. AI is no longer an experimental technology reserved for innovators, it’s rapidly becoming part of everyday business operations. 


Yet adoption and transformation aren’t the same thing. 


McKinsey’s latest global research found that while nearly nine out of ten organizations now regularly use AI, only 39% report seeing an enterprise-level EBIT impact attributable to it. In other words, most organizations have already crossed the adoption hurdle, but many are still struggling to unlock meaningful business value. 


At first glance, it really doesn’t make sense. Today’s AI models are more capable than ever. They can reason, write, summarize, analyze documents, generate code and answer questions in seconds. Every few months they become faster, cheaper and more accurate.


Yet, after building enterprise AI platforms across industries, we’ve noticed one consistent pattern:
The organizations that succeed usually aren’t the ones with access to better AI. They’re the ones that understand their business better. 


To understand why, you have to look beyond the model itself. What users experience as “AI” is really the final layer of a much larger system, and much of what determines its success happens before the model ever generates a response.

Building the AI Is the Easy Part

Ask someone what it takes to build an AI product and the answer often starts with the model: GPT, Claude, Gemini, or whichever model currently leads the benchmarks.

In a production system, the model is just one component.

What looks like a simple interaction to the user sits at the end of a much larger system:
Business problem → data → ingestion → retrieval → context → model → guardrails → integrations → permissions → evaluation → monitoring → feedback

The model may generate the final response, but every layer around it determines whether that response is useful, secure and trustworthy.


This is something we’ve seen repeatedly while building enterprise AI systems. Connecting a model can represent a relatively small portion of the actual engineering effort. The harder work is deciding what information the system should trust, keeping that information current, retrieving the right context for each question, respecting user permissions, integrating with existing systems, handling uncertainty, evaluating output quality and meeting security and compliance requirements.


And many of those challenges aren’t really AI problems.


Point AI at a real organization and it often exposes things the business has been working around for years. Four versions of the same policy across different systems, undocumented exceptions to a process, conflicting information between departments, or a workflow that functions because one experienced employee knows what to do when the written procedure stops being useful.


Point AI at a real organization and it often exposes things the business has been working around for years. Four versions of the same policy across different systems, undocumented exceptions to a process, conflicting information between departments, or a workflow that functions because one experienced employee knows what to do when the written procedure stops being useful.


Humans navigate these gaps almost invisibly. We simply ask a colleague, recognize that a document looks outdated. We remember an exception or apply context that was never formally written down.


An AI system doesn’t inherit that institutional understanding simply because it has access to your files.


Before AI can use your business knowledge reliably, that knowledge has to be made reliable, and the path it takes from source to answer matters just as much as the knowledge itself.

Every Enterprise AI System Has a Knowledge Supply Chain

Think of that path as a supply chain:

Source → ingestion → processing → retrieval → context → model → answer → user


A defective component entering a factory doesn’t become less defective because the assembly line downstream is sophisticated. Enterprise AI works much the same way. 


Its knowledge moves through a chain: 
The model sits near the end. An error anywhere upstream can therefore surface as an apparently intelligent answer downstream.


Imagine an employee asks an internal HR assistant: How many vacation days can I carry into next year?


The organization has a current HR policy saying five days. An old employee handbook says ten. A SharePoint page that hasn’t been updated in three years says seven.


All three are legitimate company documents.

Which one should the AI believe?


Semantic similarity can’t answer that question. Neither can a better prompt. The organization first has to decide which source is authoritative, encode that decision into the system and ensure outdated material isn’t treated as equally valid evidence.


This gets harder when the most valuable knowledge isn’t documented at all.


In one complex software platform we’ve worked on for years, important business rules emerged gradually through customer feedback, developer discussions, product decisions and conversations with domain experts. There was no master document containing “how the system works.” The product itself was partly the accumulated memory of hundreds of decisions.


Humans are remarkably good at operating around this kind of institutional knowledge. Someone simply asks the colleague who’s been there for six years.


An AI system can’t ask the room what everyone meant three years ago.


Before it can reason over organizational knowledge, that knowledge has to become accessible to it.


That is why plugging an LLM into a folder of company documents is not a knowledge strategy. You need to understand the entire supply chain that determines what the system knows, how it finds it, which information it trusts and what happens when the answer isn’t there.


And the first requirement of that supply chain is obvious: the information flowing through it has to be good.

What Does “High-Quality Data” Actually Mean for AI?

The problem is that “good data” means something different when AI is involved.

“Garbage in, garbage out” predates generative AI by decades. The principle hasn’t changed, but enterprise AI makes the consequences harder to spot.

A spreadsheet formula usually fails predictably. An LLM can take incomplete or outdated information and turn it into a beautifully written answer.

So “clean data” is not enough.

When we’re assessing a knowledge source for an AI system, there are several questions worth asking.

• Accuracy: Is the information correct?
• Currency: Is it still correct today? Who updates it when the underlying policy, product, price or process changes?
• Completeness: Does the source contain enough information to answer correctly, including exceptions, qualifications and edge cases?
• Consistency: Does another approved source contradict it?
• Authority: If two sources disagree, which one wins?
• Traceability: Can the system—and ultimately the user—show where the answer came from?
• Accessibility: Is this information available to this particular user, or is the AI exposing something they shouldn’t be able to see?

These questions sound more like information governance than artificial intelligence.


That’s precisely the point.


AI data quality is also more complicated than database quality. A document can be perfectly accurate and still perform badly in an AI system.


For retrieval-augmented generation (RAG), enterprise content has to be ingested, processed and indexed so that the relevant information can be found when a user asks a question. A 100-page policy manual may need to be divided into useful passages. Tables may need different handling from prose. Metadata may determine which department, jurisdiction, date or product a passage belongs to.


Get that layer wrong and the right answer may exist somewhere in your systems while remaining effectively invisible to the model.


A useful diagnostic is therefore to stop asking: “Do we have the data?”


and start asking: “Can the right information reliably reach the model, at the right time, for the right user?”

Because having good information is only half the problem. The system still has to find it.

Retrieval Is Where Good Data Becomes a Good Answer

In a RAG system, a user’s question doesn’t simply go to the language model. The system first searches approved organizational knowledge for relevant information. That information becomes context for the model’s response.


If retrieval fails, the quality of everything downstream is already compromised. 


Suppose the user asks about an obscure exception buried on page 73 of an operating manual.


The source document can be accurate. The model can be excellent. The prompt can be carefully engineered. 


None of that matters if the retrieval layer selects pages 12, 18 and 40 instead. 


This is why mature AI teams evaluate retrieval independently from generation. Did the system find the correct material? Did it rank the authoritative source above an obsolete one? Did it retrieve enough context to capture an exception? Did access controls filter the information before it reached the model?


This distinction becomes especially useful when something goes wrong. 


If an answer is incorrect, “the AI got it wrong” tells an engineering team almost nothing.


Was the source wrong?
Was the current source never ingested?
Was the relevant passage indexed poorly?
Did retrieval rank the wrong document higher?
Did the correct context reach the model, but the model interpret it incorrectly?
Did the model generate a claim that wasn’t supported by the context at all?

Those are different failures with different fixes. 


Treating all of them as “prompt problems” is like responding to every check-engine light by changing the steering wheel. 


But even when the system finds the right information and generates the right answer, there’s another question: how does the user know they can trust it?

Trust Has to Be Engineered

In enterprise AI, accuracy and trust aren’t quite the same thing.

Users need to understand when an answer should be relied on, and the system needs mechanisms for handling situations where it shouldn’t answer at all. 


This changes how you design the product. 


A trustworthy enterprise system may need source citations so users can verify important claims. Retrieval should respect role-based permissions. Sensitive operations may require additional controls. Responses should remain grounded in approved information. An important activity may need audit logging. Certain requests should trigger a human handoff.


And sometimes the correct answer is: “I don’t have enough information to answer that reliably.”

This sentence is a feature, not a failure.


One of the challenges with generative AI is that a wrong answer can sound every bit as polished as a right one. Fluency alone is not evidence of accuracy.


Good enterprise systems therefore aren’t designed to maximize how often the AI answers. They’re designed to maximize how often it should answer.


This is also where security and compliance stop being paperwork that happens before launch and become part of the architecture.


Who can access the underlying information? Where is data stored? What gets logged? Can actions be audited? What happens to sensitive information sent to external services? Which answers or actions require additional approval?


We’ve seen these questions arise repeatedly when deploying AI into organizations with serious procurement, privacy and security requirements.


If the architecture was never designed to answer them, retrofitting those requirements later can become significantly harder.


Trust is not something a model earns by sounding intelligent. It’s something the surrounding system earns by behaving predictably.


And if reliability is something we’re engineering, it also has to be something we can measure.

How Do You Know Your AI Is Actually Good?

This is where evaluation becomes critical. One of the weakest AI evaluation methods is also one of the most common. 


Someone opens the application. They ask five questions. The answers look reasonable and everyone agrees it works. 


This is not evaluation. 


Production AI needs a repeatable evaluation set built around real scenarios the system is expected to handle, including the uncomfortable ones. 


If you’re building an internal knowledge assistant, that might mean hundreds of representative questions across departments, information sources and levels of difficulty, along with expected answers or approved evidence. 


Then you can start measuring individual layers.

• Answer correctness: Did the system reach the right conclusion?
• Groundedness: Are its claims supported by the information it was given, or did it invent something?
• Retrieval quality: Did the system find the correct evidence in the first place? Did the system find the correct evidence in the first place?
• Citation accuracy: Do the cited sources actually support what the answer claims?
• Completeness: Did it omit a critical exception or qualification?
• Refusal quality: Does it decline appropriately when it lacks sufficient information?
• Task completion: Did the user actually accomplish what they came to do?
• Latency: Is the response fast enough for the workflow?
• Cost per interaction: Can this quality level be sustained economically at production volume?

Retrieval quality and response quality need to be evaluated separately. A system can retrieve the right information and still generate a poor answer. It can also produce an excellent-sounding answer from the wrong information. 


Testing only the final response hides this distinction.


The evaluation set becomes even more valuable when something changes. A team can switch models, change a prompt, modify retrieval logic or update its knowledge base and run the same evaluation suite again. 


Instead of asking, “Does the new version feel better?”, you can ask, “What improved, what regressed, and where?” 


This is the difference between experimenting with AI and engineering it. 


But even a well-engineered AI system can still fail the test that matters most: did it create any value?

How to Measure Whether AI Is Creating Business Value

A technically strong AI system is not necessarily a successful AI project. You can build a system with high answer accuracy, strong retrieval, reliable citations and fast response times, but still create very little value for the business. 


The distinction matters because AI teams and business leaders naturally look at different signals. Engineers need to know whether the system works. Leadership needs to know whether it changed anything that matters. 


The most useful way to connect the two is to measure AI at three levels.

1. System performance: Does the AI work reliably?

Is the system retrieving the right information? Are answers accurate and grounded in approved sources? Does it know when not to answer? Is it fast enough to be useful?
These metrics tell you whether you’ve built a good AI system. They don’t yet tell you whether you’ve built a valuable one.

2. User behavior: Is it improving how people work?

Once the system is technically reliable, look at what happens around it.
Are employees repeatedly using it? Are customers resolving questions without escalating to a person? Are users completing the intended task? Are they accepting answers or reverting to the old workflow? 


Adoption alone can be misleading. Thousands of interactions sound impressive, but usage doesn’t tell you whether the AI eliminated work, created new work or simply added another tool people have to check.

3. Business outcomes: Did the problem actually get smaller?

This is ultimately where AI success should be measured. If the original goal was to reduce support demand, did ticket or call volume fall?
If employees were spending too much time finding information, how much did time-to-answer decrease? 
If AI was introduced into a document-heavy process, did processing time fall? Did throughput increase? Did errors decline?
If the goal was commercial, did conversion, sales productivity or cycle time improve? 


The right metric depends on the problem. That’s why measurement should begin before development, not after launch. Establish the baseline, define the outcome you’re trying to change and then measure whether the system actually moved it. 


This is also why starting with “Where can we use AI?” is the wrong question. A better starting point is one we return to when evaluating AI opportunities: What expensive problem are we trying to solve? 


McKinsey’s research reinforces this distinction. Among the organizational practices it studied, redesigning workflows had the strongest relationship with reported EBIT impact from generative AI. The value was in changing how work got done, not simply introducing AI. 


That leads to a much higher bar for success. Don’t ask only: Is the AI performing well? Rather ask: Is the business performing better because of it?
Because ultimately, that’s what separates an AI capability from a competitive advantage.

The Competitive Advantage Isn’t the Model

The models will keep getting better. They’ll become faster, cheaper and more capable. Features that once required specialized engineering will become available through APIs and platforms. Today’s model advantage will eventually become tomorrow’s commodity. 


But the rest of the system won’t commoditize at the same speed.


Your proprietary knowledge. Your understanding of which sources can be trusted. Your integrations and workflows. Your permissions and governance. Your evaluation datasets. The failure cases you’ve discovered. The feedback loops you’ve built. Your understanding of where customers struggle and where employees lose time. 


Those things accumulate. And this may be the most important shift for leaders to understand about enterprise AI. 


The competitive advantage is moving away from simply having access to intelligence and toward how effectively an organization can connect that intelligence to its own knowledge, systems and work. 


Two companies can have access to the same models.
They will not build the same AI, because the model is only one ingredient.
The business around it is the system.