“The companies seeing the biggest return from AI aren’t better at measuring after deployment. They’re better at defining success before deployment.”

Everyone is asking the wrong question.

After six months of an AI initiative, leadership teams gather in steering committees asking, “Is it working?”

Someone presents adoption statistics.

  • 7,000 employees have access.
  • 3,400 prompts were generated yesterday.
  • Usage increased 42%.
  • Satisfaction scores are improving.

Then someone asks the question that matters, “Has the business actually improved?”

Silence. Nobody can answer. Not because AI failed, because nobody defined what success looked like before they started.

That’s the mistake.

The measurement problem doesn’t begin after AI deployment. It begins before the first prompt is ever written.

AI Doesn’t Create Measurement Problems. It Exposes Them.

One of the biggest surprises working with executive teams over the last two years is that AI rarely introduces new management problems. Instead, it exposes the ones that were already there.

Teams struggle to prove AI value because they were never measuring the underlying work in the first place.

If you don’t know:

  • how long a decision currently takes
  • where work gets stuck
  • who owns the outcome
  • what “good” looks like

then AI can’t magically create a measurable return on investment. You’re simply automating undefined processes.

The evidence is becoming remarkably consistent. For example, McKinsey’s global AI survey found that although AI adoption is now widespread, more than 80% of organizations still report no tangible enterprise-level EBIT impact from generative AI. Only 17% attribute 5% or more of EBIT to AI initiatives. McKinsey & Company

MIT’s NANDA Initiative paints an even starker picture. After studying more than 300 enterprise AI deployments, it found that around 95% of generative AI initiatives failed to produce measurable profit-and-loss impact, with only about 5% successfully scaling into meaningful business value. Samson Redline Technologies

These 2025 findings gave us a useful baseline for the gap between AI adoption and measurable enterprise value.

A year later, the same challenge can be seen in banking. In July 2026, Evident assessed the publicly available AI activity of 20 major banks in Latin America. Seventy percent published AI use cases with reported outcomes, while 55% disclosed how many use cases they had in production. Yet none publicly reported projected or realized ROI across all their AI activity.

Evident AI Index Banks – LATAM

Evident uses an outside-in methodology based on publicly available information. The findings do not tell us what these banks measure internally. Although the assessment is limited to banking, it shows how organizations can report individual AI outcomes while still lacking a clear view of what those gains add up to across the enterprise.

It’s also why we started running the Artificial Organizations AI Executive Study each year. The 2026 AI Executive Survey will help us understand what organizations are measuring today and how they are connecting AI initiatives to business outcomes and ROI.

Start With the Workflow, Not the Tool

Let me share a story from a recent executive program, which I’m sure you’ll all recognize. I see this every week. The majority of AI projects begin like this.

“We should use Copilot.”

“We should deploy ChatGPT because it’s better.”

“We need to build an agent.”

Those aren’t business problems. They’re purchasing decisions. They’re a tool-first approach to innovation, which only creates more noise, more output and no transformation.

Instead, start by asking: Which workflow are we trying to improve?

Examples might include:

  • preparing executive board papers
  • reviewing customer complaints
  • approving credit decisions
  • onboarding employees
  • producing weekly operating reviews
  • responding to RFPs

Now the conversation changes.

Instead of discussing AI capabilities, you’re discussing business performance.

That is where measurement begins.

Value Stream Mapping (3)

Every Workflow Needs Five Focusing Questions

Before introducing AI, every workflow should answer five simple questions.

1. What outcome are we trying to improve?

Not activity. Not a tool to use. The outcome you’re aiming to achieve.

Examples include:

  • reduce decision cycle time
  • improve customer satisfaction
  • increase forecast accuracy
  • shorten onboarding
  • improve first-call resolution
  • reduce regulatory risk

If the outcome isn’t explicit, neither is success.

2. What is today’s baseline?

This is where many organisations struggle. Everyone wants to measure improvement. Very few know where they started.

For example:

Current board paper preparation:

  • 12 hours
  • 4 contributors
  • 3 review cycles
  • 392 pages in the final pack

Current hiring process:

  • 42 days
  • 18 approvals
  • 27 emails

Without today’s numbers, tomorrow’s improvement is just opinion.

3. Who owns the result?

One of the easiest ways to kill an AI initiative is shared ownership.

IT owns the platform. Operations owns the process. Business owns the budget. Nobody owns the outcome.

Successful AI programmes always have a single accountable business owner, and a clear cross-functional team working together to solve it.

Not to agree on what technology to use, but to agree, align and redesign the solution for the business result.

4. What evidence will prove success?

This is where leaders should become much more disciplined.

Instead of saying, “We’ll know when we see it.”

Define the evidence beforehand.

For example:

  • preparation time reduced by 40%
  • decisions made in one meeting instead of two
  • customer response time reduced by two days
  • proposal win rate increased by 15%
  • employee onboarding reduced from four weeks to two

Notice these are business measures, not “AI measures”.

5. How often will we review it?

Measurement isn’t an annual exercise. It must become part of the operating rhythm.

Weekly, monthly, quarterly… whatever aligns to your decision-making cycles as leaders.

If leaders only review AI performance every six months, they’ve already lost six months of learning.

If uncertainty is high, have shorter review cycles. As you get to consistency and predictability, move to longer cycles. Even better, set guardrails or boundaries for how the systems should operate, then let the machine tell you when the system is operating outside them, or trending towards breaking them!

Don’t Measure AI.

Measure Better Decisions.

One idea from Artificial Organizations keeps proving itself across leadership teams I give keynotes or workshops to, or those that take our coaching programs.

AI is a judgment infrastructure rather than a mere productivity tool. That insight fundamentally changes what you should measure.

Instead of asking, “Did AI generate the report?”

Ask:

  • Did leaders make the decision faster?
  • Were fewer decisions revisited?
  • Were risks identified earlier?
  • Was less time spent rebuilding context?
  • Did teams align sooner?

Those are signs that judgment is improving. And judgment is where competitive advantage compounds.

An Illustrative Scenario

Imagine two companies introducing AI into their weekly executive review.

Company A measures:

  • prompts generated
  • licences activated
  • meeting summaries created

Company B defines success before deployment.

Outcome: Reduce executive decision cycle time.

Baseline:

  • Four hours preparing.
  • Two-hour meeting.
  • Three follow-up meetings.

Target:

  • Preparation under one hour.
  • Meeting reduced to ninety minutes.
  • All decisions closed during the meeting.

After eight weeks, they can clearly demonstrate:

  • 60% reduction in preparation time
  • 40% shorter meetings
  • 70% fewer deferred decisions

Same AI.

Completely different management discipline.

Company A is still talking about what activity they did.
Company B can explain how performance improves, return on investment, and final outcome.

This is why you need to stop measuring activity and start measuring outcomes.

AI Governance Begins Before Deployment

As organizations mature, governance should evolve beyond security and compliance.

Leadership teams should ask every AI initiative the same questions before approving funding.

  • What workflow are we improving?
  • What outcome are we targeting?
  • What’s today’s baseline?
  • Who owns the business result?
  • How will we know this worked?

If those questions can’t be answered, the project isn’t ready.

Not because the technology isn’t mature. Because the operating model isn’t.

Side note: If you wish to understand AI governance, operating models and how to measure success in greater detail, I’m delighted to share that John Marcante, former CIO of Vanguard, and I will be publishing a 5–part series on these topics over the next few weeks. Make sure you sign up to the newsletter to not miss out.

The New Leadership Discipline

The organizations creating the most value from AI are remarkably consistent.

They don’t start with the technology. They start with operating discipline.

They define:

  • the workflow
  • the baseline
  • the owner
  • the outcome
  • the evidence

Only then do they introduce AI.

That small shift changes everything including your perspectives, the activities you do, and the final outcome you achieve.

My guidance to you is before approving your next AI pilot, ask one simple question,

“If this succeeds, exactly what will be different in our business, customers or team’s world? And how will we prove it?”

If you can’t answer that before deployment, you almost certainly won’t be able to answer it afterwards.

Help Shape the 2026 Artificial Organizations AI Executive Study

One of the biggest gaps in AI today isn’t adoption.

It’s evidence.

This month we’re launching the 2026 Artificial Organizations AI Executive Survey, building on the research behind Artificial Organizations.

Last year, more than 5,000 executives helped us understand how leaders were adopting AI. This year we’re focusing on a harder question:

How do organizations actually measure AI outcomes and ROI?

We’re looking at questions such as:

  • Which AI initiatives are producing measurable business value?
  • How are leaders measuring decision quality, speed, and outcomes?
  • What metrics are boards actually asking for?
  • What separates organizations proving AI ROI from those still chasing adoption?

If you’re an executive leading AI transformation, I’d love you to take part.

2026 Artificial Organizations AI Executive Survey

Join the newsletter below, and you’ll receive:

  • Early access to the research findings
  • An invitation to participate in the survey
  • Practical frameworks for measuring AI outcomes inside your organization

Because the future won’t belong to the organizations using the most AI.

It will belong to those who can prove it’s creating better outcomes.

Sign-up to our newsletter to take part!

FAQ

1. Why should organizations define AI outcomes before deployment?

Without a clear outcome and baseline, leaders cannot tell whether AI improved the work or simply increased activity. Define the workflow, expected result, accountable owner, and evidence of success before deploying the technology.

2. What should executives measure instead of AI adoption?

Adoption is an input, not an outcome. Executives should measure changes in business performance, such as decision speed, customer satisfaction, cost, revenue, productivity, forecast accuracy, risk, or decision quality.

3. What five questions should every AI initiative answer?

Before implementation, leaders should ask:

  • What business outcome are we trying to improve?
  • What is today’s baseline?
  • Who owns the business result?
  • What evidence will prove success?
  • How often will we review progress?

If these questions cannot be answered, the initiative is not ready.

4. What will the 2026 Artificial Organizations AI Executive Survey examine?

Building on research involving more than 5,000 executives, the survey will explore what organizations are measuring today, which AI initiatives are producing meaningful outcomes, and how leaders connect those outcomes to business performance and ROI.

The findings will inform the 2026 Artificial Organizations AI Executive Study. Join the newsletter to participate and receive the findings.

References