AI Reality Check: Leaders Rate Their Orgs Highly on AI Maturity But Results Tell a Different Story

Many companies, led by their senior executives, are over-confident about their positioning when it comes to use of AI. The reality of how they’re actually transforming their business  doesn’t come close to their level of optimism.

That’s a big takeaway from newly released Avanade research analyzing AI progress among 3,600 companies.

The data delves into underlying reasons for the lack of progress on building AI use cases that truly change the business: a focus on internally driven AI deployments rather than external, customer-focused use cases, the need to bolster security and privacy for AI, and shadow AI, to name just three.

 In a conversation before the release of Avanade’s data, the company’s top AI leader highlighted today’s disconnect between adding AI into existing products or processes – which is what the data show most companies doing today — vs. fundamentally rethinking and redesigning the ways things work.

“The biggest trap we’re seeing clients fall into with AI transformation is bolting AI onto processes that were designed for a world without it,” said Ben Beath, Chief AI Officer at Avanade. “The real value comes when you’re able to redesign an end-to-end workflow around what AI and agents and people each do best.”

This approach can be applied to processes like procure to pay and customer service, Beath added. “They’re places where we see our clients fundamentally rethinking how they how they do these processes using AI.”

Overconfidence Problem

Here are a couple data points indicating how companies are rating their own AI results: 72% cite high confidence that their teams are using AI effectively; unfortunately, 75% say their leaders are overstating their AI maturity.

There’s more: 81% of senior decision makers have complete or high confidence that their team is using AI effectively, while 76% believe the organizations they lead are more advanced than their competitors. But on the ground – managerial levels below top execs — the view is less upbeat than that of the most senior leaders. On the same two points, 63% of middle and junior managers believe their team is using AI effectively, and 60% of that second category believe they’re more advanced than competitors.

Several other findings from Avanade show how far companies have to go on the path to AI-powered business transformation:

• 66% of respondents have yet to formally calculate return on investment (ROI) from AI and to have the results reviewed by finance or senior leaders. That means those senior leaders discussing powerful results lack the data to make the claims or to back them up.
• 18% — less than 1 in 5 — say they’ve fundamentally reshaped how their entire team operates around what AI makes possible, while their actual focus is on streamlining existing processes and boosting productivity. The latter are worthy goals, of course, but not game changers by any measure.
• A mere 6% report their organization’s AI focus is on creating new operating models, revenue streams, or businesses. That’s aiming low.  

One reason that companies are overly optimistic about their level of AI aptitude and results may be what financial analysts would call “easy comps” or comparisons. That is, they may be looking at traditional competitors that are likely in a similar stage of AI transformation journey as their firms, rather than comparing with nimble innovators that are likely better positioned to shake up industries.

Just 28% — less than 1 in 3 — of survey respondents say they use AI-native firms as a measuring stick for their AI progress. That’s surely a way to cast your company in a more favorable light, but that light could dim quickly.

Internal Focus and Other Barriers

There’s a fascinating – and nuanced — set of findings on use cases where transformational value is emerging. Overall levels of transformation are quite modest: all reported areas were cited by less than 25% of respondents.  

Specifically, the functions that are undergoing transformational improvement align more with revamping what exists than creating something  new. The highest figures are in areas including operational efficiency (24% of respondents) and employee productivity (23%). More externally focused transformations including product or service innovation were even lower (22%) while revenue generation and speed to market (21%) were tied below those other functions. All those figures create a clear picture of where companies are overall in transforming their business.  

Percent of Respondents Achieving Each Type of Transformational Improvement

Looking forward, companies report they’re maintaining a similar focus; that is, operational efficiency is the most commonly cited AI goal for the next 12 months, identified by 47% of respondents, followed by employee productivity at 44%. Product or service innovation comes in well below, at 35% of respondents. I think these findings help explain why 77% of respondents report that too much of their AI investment is focused on incremental improvements and not more transformational opportunities.  

There are troubling findings that suggest companies don’t yet have a complete handle on AI usage or controls. Avanade’s data shows 45% of respondents believe their staff use AI outputs without adequately verifying accuracy or suitability. There are indicators that shadow AI is also a challenge: 32% believe AI is being used in ways compliance teams would not approve if asked and 31% report the use of unapproved AI tools.

Companies do have a clear understanding of where and how they need to improve to more effectively deliver on claims of ambitious AI agendas. 70% of survey respondents said they need at least moderate increases in AI security (30% said “significant strengthening” is needed, while 40% called for moderate strengthening), while 68% cite the need for better regulatory compliance and 67% point to the need for greater privacy.

Avanade CEO Chris Howarth distilled the results into one penetrating critique: “Ambition is not the problem. The problem is that most organizations are using AI to improve a business model that AI should be replacing.”

AI is indeed replacing business models. The question is whether corporate leaders will respond to, and build for, that reality or continue to speak in platitudes because they’ve used AI to make some processes more efficient while true innovators are pursuing transformational initiatives to capture enduring competitive advantage.

Additional AI Insights:

• AI in Financial Services: Leaders Have Big AI Plans But They Need to Redesign Their Institutions
• AI in Government: Google Cloud Suite Includes Pre-Built Skills, Connections to Third-Party Systems
• AI in Healthcare: OpenAI Connects Clinicians to Patient Medical Records Insight ChatGPT