When Tokens Cost More Than People

I was talking to a CFO at a capital-heavy business that counts every dollar. His first question had nothing to do with models.

The paragraph that answered the question was indexed and sitting in our vector store.

He wanted to know how to get on top of his AI investment. What did it cost? What did it produce? And would he ever be able to see the difference?

Many AI programs cannot answer those questions.

They can tell you how many tokens were consumed, which models were called and what the cloud bill was. What they often cannot tell you is what a business outcome cost to produce.

That distinction is becoming important as AI moves from chatbots to agents.

The unit price of intelligence is falling. The cost of using intelligence at scale may not be.

You can read the bill. You can't read the cost of the work.

Companies have never had trouble finding their AI bill. Azure, AWS and other cloud platforms can show spend by service, resource and workload.

The harder question is: what did that spend buy?

What did it cost to resolve a support case? Produce a qualified lead? Review a contract? Complete a software change?

If the system used three models, made twelve calls, retrieved documents, invoked tools, retried failed steps and sent the result to a human for review, what did that outcome actually cost?

The invoice usually does not tell you.

A useful AI cost model needs to follow the work, not just the infrastructure. It needs to account for inference, orchestration, tool calls, retrieval, retries, evaluation and human intervention, and connect that total to a measurable outcome.

Without that denominator, cost is just a number.

And numbers nobody can connect to an outcome are very difficult to manage.

Cheaper tokens, bigger bills

The economics are getting stranger, not simpler.

Gartner expects inference on a trillion-parameter model to cost more than 90% less in 2030 than in 2025. That sounds like the end of the cost problem. But Gartner also expects agentic workloads to consume dramatically more tokens per task, with consumption potentially growing faster than token prices fall.

The same effect appears at the workflow level. An interaction that cost pennies when AI was answering a question can cost dollars when an agent plans, calls tools, checks its work and tries again.

But the important number is not the price of a token.

It is the cost of a successful outcome.

If a $1 interaction replaces $8 of human effort, that may be excellent economics.

If it produces an answer that still requires a person to redo the work, it may be terrible economics.

Token price alone cannot tell you which one you have.

Usage is not value

Uber provides a useful warning.

In April, Uber's CTO said the company had exhausted its 2026 budget for AI coding tools after only four months as adoption of agentic coding accelerated across engineering.

The obvious lesson is that AI can generate a surprisingly large variable bill. But more importantly: usage is not an outcome.

A team can generate more code, run more agents, and consume more tokens without creating proportionally more business value.

Uber's subsequent experience is even more interesting. By August, the company reported that weekly agentic requests had grown 9.4 times from February while total AI spending had stabilized through optimization. Cost per session had fallen 52% from its June peak through changes including model selection and routing.

That is what good AI economics looks like. Not “use less AI.” but get more useful work from every dollar of AI spend.

The same principle applies to model selection. The most capable model is not automatically the most economical model. A smaller model may be better for a routine task; a frontier model may be worth its premium when the additional reasoning materially improves the outcome.

The question is not which model is best.

It is which model is economically justified for this task.

When AI is a black box

You cannot manage what you cannot see.

An AI workflow can look simple from the outside while doing a surprising amount of work underneath: multiple model calls, retries, retrieval, tool calls, routing, evaluation and human intervention. If you only see the final answer and the monthly bill, much of the cost is invisible.

Observability changes that.

Traces can show when calls fail and get repeated, where a workflow is spending time and tokens, how much context is being passed into each call, and which models are actually doing the work. That turns an AI system from a black box into something you can manage.

Three costs are particularly easy to miss.

Retry tails. If an agent succeeds 90% of the time on its first attempt, roughly 1.1 attempts are needed for each successful outcome. At 70%, that rises to roughly 1.4. The result is that a seemingly modest drop in reliability can increase the effective cost of every successful outcome. In messy enterprise workflows, failure is not just an accuracy problem. It is an economic one.
Context inflation. More context feels safer, so systems tend to retrieve more documents, carry more history and pass larger payloads into models than the task actually requires. But context is not free. Longer inputs require more computation, and careless context growth can turn a cheap workflow into an expensive one. The question is not whether the model has enough information. It is whether it has the right information.
Routing. When teams do not know which model is good enough, they default to the strongest one. A simple classification task ends up on the same frontier model used for complex reasoning. At millions of calls, that difference is enormous. Routing the right task to the right model can be the difference between a manageable AI program and a board-level cost problem.

The goal is not to make every AI workflow cheaper.

It is to make its economics visible to make deliberate tradeoffs: where to spend more for quality, where to use a smaller model, where to reduce context, and where a workflow is simply not worth running.

You cannot optimize a black box.

Customers don’t buy the AI for AI

There is another mistake companies make when they build the business case: They ask whether customers will pay extra for the AI feature.

Usually, they won't.

Customers care that a problem was solved faster. That a decision is better. That a risk was reduced. That they did not have to hire another person to perform repetitive work.

They pay for outcomes, not tokens.

So the business case needs to be expressed in the same language as the cost.

If an AI workflow costs $0.40 per completed case and saves $2 of labor, that is a useful number.

The metric has to capture the outcome, not just the activity.

MIT NANDA's 2025 study reported that 95% of the organizations in its research sample saw no measurable P&L impact from their generative AI initiatives. The study was preliminary and its sample was not intended to represent every company, but the warning is important: deploying AI is not the same as producing measurable economic value.

Give every experiment a reason to survive

Organizations get real value from AI by treating early initiatives as experiments in business economics, not just technology.

That means defining the baseline first. What does the work cost today? How long does it take? What quality level is acceptable? What percentage can AI complete without intervention? What will the workflow cost at the volume we expect? And perhaps most importantly: : What would make us stop?

An experiment that proves a use case does not create enough value has succeeded. It prevented the organization from spending more money on a bad idea.

A production launch that discovers the same thing six months later is much harder to shut down.

This is where AI programs often develop the wrong incentives. Teams are rewarded for getting technology into production, not for determining whether it deserves to stay there.

A mature AI portfolio needs both value metrics and economic metrics:

Value: revenue, time saved, quality, cycle time, risk reduction, capacity created.
Economics: cost per task, cost per successful outcome, model mix, consumption, failure rate and human intervention.

You need both.

Otherwise, you can optimize the cost of an outcome without knowing whether the outcome matters, or maximize an outcome without knowing whether you can afford it.

Keep the platform. Change the economics.

None of this necessarily requires replacing the technology you already own. If you’ve invested in a cloud platform, don’t tear it out, but make the economics visible and controllable: route routine work to cheaper models when quality allows but use a frontier model when the added reasoning earns the premium, detect runaway workloads, measure the cost of a successful outcome rather than the number of tokens consumed, and continuously compare that cost with the value the workflow produces.

The fundamental problem is not that tokens are expensive.

It is that AI makes the cost of doing work variable, while many organizations still manage it like a fixed technology expense without associating the value being generated by those tokens.

So, back to the CFO's question; - What did we spend, and what did we get?

For some workloads, the tokens already cost more than the people they were supposed to help. For others, AI is dramatically cheaper. You cannot know which is which, from the model invoice.

You know by measuring the economics of the work itself.

And sometimes the most valuable AI decision is not which model to deploy but deciding not to deploy one at all.