Insights
From token sprawl to cost discipline: The 2026 executive view of LLM economics
Adnan Masood, PhD, Chief AI architect, UST.
Enterprise AI economics now cut against an old instinct. Unit prices keep falling and total cost keeps rising, because usage is growing faster than price is dropping and most of that usage runs unchecked.
Adnan Masood, PhD, Chief AI architect, UST.
Token cost is no longer a line item you reconcile after the invoice arrives. It is an architectural property of the systems you build, and it has to be governed like one.
Most enterprise AI budgets were built on a quiet assumption: the price of a model call keeps dropping, so the total bill will look after itself. The first half is true. Per-token prices have fallen steadily. The trouble is that usage has climbed much faster than prices have come down, so the total goes up even as each unit gets cheaper. The space between what one call costs and what a running system costs is where the money is going.
The headlines of 2026 tell the story better than any forecast. Uber spent its entire full-year budget for AI coding tools in four months, after putting engineers on internal leaderboards that ranked them by how many tokens they used and then capped each employee at $1,500 a month per tool to stop the bleeding. Its own COO admitted he still can't draw a credible line between the rising Claude Code bill and the features that actually ship to customers. An AI consultant told Axios that one of their clients spent half a billion dollars on Claude in a single month after forgetting to put usage limits on employee licenses. Microsoft started pulling most of its direct Claude Code licenses in favor of a cheaper route. In every one of these cases the technology did exactly what it was supposed to. What broke was the control around how much of it people used.
DIVIDER
Why the old cost model broke
Companies used to treat AI spend like any other software license: buy a tool, pay per user, and watch the bill grow in step with headcount. That model has come apart in three ways.
The first thing that changed is what you pay for. Software was sold by the seat. AI is sold by the token, which is roughly a few characters of text the model reads or writes. Token use is hard to predict because it depends on how long the prompt is, which model handles it, and how often the system loops back over its own context. Two people doing the same job can differ tenfold in what they cost.
The second thing that changed is the work itself. A traditional application makes one call and returns an answer. An agent plans, looks things up, calls tools, checks the result, and tries again, often burning through something like a thousand times the tokens of a single question to finish the same task. A feature that looked cheap in a demo becomes a standing expense the moment it starts running on its own at scale.
The third thing that changed is that nobody could see where the money went. When you rank people by how many tokens they burn, heavy usage starts to look like hard work, and the two are not the same. Amazon reportedly shut down its internal usage leaderboard after staff were caught running busywork to climb it. If you can't connect a dollar of spend to a team, a feature, or a result, you can't manage it, and you certainly can't defend it when the board asks what you got for the quarter.
The takeaway is simple. Cost has stopped being a finance question you answer after launch and become an architecture question you have to answer before it.
DIVIDER
The four cost surfaces of enterprise LLMs
It helps to stop treating "the AI bill" as a single number. Cost actually gets created in four different places, each for its own reason and each needing its own fix. The common error is to argue about the one you can see, the price per token, while the other three quietly run up the total.
This is the published price per million tokens, the figure everyone quotes, and it matters least to the final bill. The waste here is rarely the rate. It is the mismatch: using a top-tier model for a job a cheaper one would finish, sending a long system prompt on every single call, and paying full price over and over for context that never changes between requests.
The fixes are familiar and oddly neglected. Send easy work to smaller models and save the expensive one for the hard cases. Turn on prompt caching for the parts of a prompt that stay the same, such as standing instructions and reference material, so you pay to store them once and then read them back at a fraction of the cost. Move anything that doesn't need an instant reply, like overnight document processing or offline evaluations, onto the batch tier at half price. Stack caching and batching together and you can cut a workload's effective cost by most of an order of magnitude. None of this is advanced. Most companies simply haven't switched it on.
Surface 2 | Tools and seats: the developer copilots
This is where most 2026 budgets came apart. Coding assistants like Claude Code, Copilot, and Cursor charge for the seat and then meter the tokens on top, and by default that meter has no ceiling. Push adoption hard, add a leaderboard, and you get Uber's outcome: a year's budget gone in a third of the year.
The fix sits at the point of use. A plain monthly cap per person per tool, which is what Uber put in place after the damage was done, turns an open tap into a fixed allowance. The harder part is cultural. Stop celebrating raw consumption. Tokens spent measure effort, not results, and grading engineers on how many they burn teaches the whole organization to spend for the sake of spending.
Surface 3 | Agents: the meter that compounds
Agents are the most likely source of the next overrun, because their cost builds up where you can't easily watch it. A single agent run can spin out into dozens of model calls, retries, tool invocations, and repeated passes over the same context. That thousand-to-one multiplier isn't a freak event. It's how an unconstrained loop normally behaves.
The fix lives in the design, not the budget line. Agents need step limits, loop guards, and a hard cap on how many calls a task can make, enforced while they run rather than discovered on the invoice. Let an agent earn more room only once it has shown that the extra steps are worth the spend. Run each one like a small cost center with a named owner for its budget, instead of shipping it like a feature with a switch.
Surface 4 | The hidden tax: redundancy, sprawl, and shadow AI
The last surface never shows up as its own line. It's the retrieval step that crams ten passages into context when three would do, the duplicate calls, the chat histories that grow without limit, the tools a team signs up for without telling procurement. Each piece is small on its own. Together they decide whether your bill is something you can forecast or something that ambushes you.
The fix starts with being able to see it. You can't trim what you can't measure, and you can't optimize what you can't trace back to a source. The single most-requested capability among FinOps teams this year is exactly this: usage broken out by token, request, and GPU, because most organizations still can't say which of their AI features earn back more than they cost to run.
DIVIDER
The operating model to put in place now
Chasing savings one trick at a time is the slow way to lose ground. The faster route is to run tokens the way good teams already run cloud, with visibility, attribution, optimization, and accountability built in from the start. In practice that comes down to five moves.
First, instrument before you optimize. You can't manage what you can't attribute. Put token-level tracking in place that ties every call back to a team, a feature, and where possible a customer, and adopt a shared standard such as the FinOps Open Cost and Usage Specification so AI cost lands in the same dashboards as the rest of your cloud spend. The companies that build this baseline now will understand their own economics for years. The ones that put it off will keep reacting to invoices after the money is already gone.
Second, enforce at the gateway, not the invoice. The most useful single decision is to route all model traffic through one gateway. An open-source layer like LiteLLM gives you a single interface to every provider, and a layer like Portkey adds routing, caching, and spending limits on top. Timing is the whole point. A budget rule checked at the gateway can stop an overspending call before it ever reaches the provider, rather than triggering an alert on the 28th of a month you've already blown.
Third, make the efficient path the default. Caching, batching, and model tiering shouldn't depend on a developer remembering to use them. Wire them into the standard route so the cheap option is also the easy one. If a routine workload isn't already caching its fixed context and reaching for a right-sized model on its own, the platform is paying more than it has to.
Fourth, treat agents as cost centers. Every autonomous workflow gets an owner, a cap per task, and a check on its output before you grant it more autonomy or a larger budget. The question to ask is never whether the agent can do the work. It's whether each extra step it takes is worth what it costs.
DIVIDER
Six questions every CxO should ask before scaling LLM spend
- Can we trace every dollar of AI spend to a team, a feature, and an outcome, close to real time?
- Where does enforcement happen, at the gateway before the call or on the invoice after the month?
- Are caching, batching, and model tiering built into the default path, or left to individual teams to remember?
- Which of our agentic workflows can spend without a ceiling, and who owns that budget?
- Are we rewarding token consumption anywhere, openly or by implication, and what should we measure instead?
- For our highest-spend features, do we know the cost per unit of value, and is that ratio improving?
If the honest answer to most of these is no, the problem isn't the price you're paying. It's the discipline you haven't built yet, and discipline is buildable.
DIVIDER
Closing perspective
Enterprise AI economics now cut against an old instinct. Unit prices keep falling and total cost keeps rising, because usage is growing faster than price is dropping and most of that usage runs unchecked. The companies that struggled in 2026 weren't overpaying per token. They had no way to see, attribute, or limit how those tokens were spent.
The teams that pull ahead over the next couple of years will handle token economics the way the best of them already handle explainability: as something designed into the system from the start, not a report assembled after the fact. Visibility, enforcement, efficient defaults, and accountability are not cost-cutting gimmicks. They are what separates an AI program you can grow from one that grows past your control.
If your AI spend is growing faster than your ability to explain it, we would welcome the chance to help you bring it back under control.
UST's AI and Responsible AI teams help enterprises build the gateway architecture, FinOps tracking, and agentic guardrails that turn an unpredictable invoice into a number you can defend.
Connect with an expert to review and learn more