Agentic Reasoning Loops Could Drive a 17x Increase in AI Token Consumption

Welcome to the Cloud Wars AI Minute — your AI news and commentary show. Each episode provides insights and perspectives on the AI revolution, exploring the technologies, companies, and leaders transforming business and reshaping the future.

In today’s Cloud Wars AI Minute, I break down the rise of agentic reasoning loops and why their impact on AI token consumption and costs could be significant.

Highlights

00:12 — Today’s topic is going to be agentic reasoning loops. Everyone’s moving to this concept where we can have a planning, act, observe, and reflect type of process, which allows us to be able to get much deeper analysis and much more resilient implementations across AI, across the world.

01:03 — Now, the other thing is with this, we’re seeing that about 33% of enterprise software will run on this kind of approach by 2028. Also, the other part of this is that because these different loops are taking place, we’re actually seeing that tokens are going to get more and more consumption happening off the back end.

01:38 — So one of the driving factors of this is going to be the increase that we’re seeing, and just to give you perspective of what we’re starting to see, it’s about a 17x on a single chapter that we’re starting to see happen as a result of this. So what it used to do when we just did simple RAG patterns.

02:01 — Now we’re seeing about 17 times the amount of tokens being consumed, and when we start seeing that level of token consumption, we’re going to see that while we’re getting better answers, there’s also going to be a higher cost that’s associated to it.

02:28 — But it doesn’t matter that the tokens are necessarily coming down in cost because of the fact that what we used to see is that an average transaction would be about three cents, and now we’re seeing it move to about 15 cents across the board, and so this is going to be one of those things that we really need to be thinking about as you start to build out your solutions.