The token price is not the cost of your agent

The real cost of an AI feature is the cost of a run that finishes well, not the price per million tokens. You add the context you resend, the model calls, the tools, the retries, the layer around them, and the minute of whoever has to fix it, on a specific tenant. The token is the raw material and the agent is the process.

List prices per million tokens are falling, and the agent invoice is still rising. That is not a spreadsheet error: you are measuring the raw material, not the work.

The right unit is not euros per million tokens, but the cost of a run that finishes well: a closed task, on a specific tenant, without a human having to redo it. The token is the line on the model invoice; the agent is the process.

McKinsey wrote it in July 2026 without hedging: tokens are not value, they are the bill. In their survey, 93% of respondents were over their AI budget. The Stanford HAI 2025 index documented that the cost of inferring GPT-3.5-level capability fell from $20 to $0.07 per million tokens through 2024. Over the same stretch, enterprise spend on LLMs (large language models) tripled.

That tension also reaches the committee. BCG's AI Radar 2026, with almost 2,400 executives, says companies plan to double AI investment: from 0.8% to 1.7% of revenue. Almost every CEO believes agents will produce a measurable return this year. In The State of AI 2025, 88% already use AI in some function, but only 39% attribute any impact to EBIT (operating profit). Most of that group say it is less than 5%, and only 6% report a serious impact.

Teams are spending more and expecting more from agents, but profit does not show up on its own. The difference is usually the unit you watch. We break down TCO (total cost of ownership) in how much it costs to implement an agent. Here we stay with one run: why the token price does not tell you whether the case pays.

Why token prices fall and the bill goes up

The model's list price is a raw-material rate. A one-turn chat consumes that material once; an agent consumes it in a loop: it rereads context, calls tools, corrects itself, and generates again.

That same paper lists why the deflation never reaches the invoice. Context is resent at every step, because models do not remember on their own. Refining the answer (checking, repairing, and verifying again) eats around 60% of an agentic task's cost, and the same task can cost 30 times more depending on the path the agent takes. Expensive reasoning gets used for easy work, and orchestration (how tools and handoffs are split) multiplies spend without improving the outcome. The shape of the information matters too: the same meaning in a non-English language often splits into more tokens than English.

That is why you see headlines that "the token is on the floor" and a CFO asking why this month's line tripled. They are not contradicting each other: they measure different things. The economics belong to the workflow, not the rate card. If you do not instrument the run, the price per million is a comfort. There is a practical guide on where agents pay off if you want the long breakdown.

What makes up the cost of a run?

A run is the full trip to an acceptable task, not "a prompt". In a SaaS it usually has these pieces.

Context. Ticket, history, documents, tool schema, and instructions. You pay it on the way in and you pay it again every time the agent takes another step.

Model calls. Input tokens, output tokens, cache if you have it, and reasoning if the model "thinks" before it acts. Treat Claude's public rates as a floor, not a cap. The real price rises with a long window and with retries.

Tool calls. Every time the agent touches your API, an MCP (a protocol to connect the agent to tools), or a search index, you pay orchestration inference plus the infra of that call. If the tool returns a huge dump, that dump enters the next context.

Retries and refinement. If the tool fails, the JSON does not validate, or the agent does not trust itself and asks again, this is where a large part of the bill nobody budgeted lives.

The layer around it. Gateway, logs, evaluation, and queues. Some of the fastest-growing spend is no longer the LLM: it is security, orchestration, and monitoring that also charge per token or per trace.

Human. Review, correction, and escalation. If the agent drafts and a person sends, that minute belongs in the cost of the run. If the agent is wrong, you pay the minute and the rework.

Add that up and stop asking "how much is a million tokens?". The useful question is "how much does it cost for this ticket, in this tenant, to come out right?".

Hidden costs

The average lies, because cost behaves like a distribution rather than a fixed unit price. The 5% of runs that go off the rails (long context, eight tools, a loop) can eat half the month's invoice.

The fat tenant hides too. A customer with an endless history and badly cut documents does not cost "a bit more": it is a different species of run. If you average it with the clean tenant, the one that destroys margin never shows up.

The person who supervises and corrects an agent's mistake does not appear on the model invoice. A failure charges you cents of API and minutes of someone who has to rewrite the result.

There is also a cost to missing stop rules and a retry cap. Without a spend limit per thread and per tenant, an agent can repeat tools until a cent becomes a night's invoice. You do not need a famous incident: you need an agent with no stopping rule. We have seen it more often than we like to admit.

And there is a language cost. If your product is not in English, do not copy a budget built for English: the same meaning tokenizes worse. The method does not change, the number does.

Metrics that help

Three numbers are enough to decide; the rest is dashboard noise.

Cost per correct task. Model + tools + retries + the human on that run, only when the result is accepted. If you do not measure failures separately, you are averaging theater.

Success rate without repair. What share comes out right without someone rewriting it. That is the lever that decides whether you can remove the human or not.

p95 per tenant. The 95th percentile, not the mean: what an ugly run costs on the fat customer. Margin breaks in the tail, not in the happy path from the demo.

Add two stress tests to those three metrics, the ones Alberto flagged in his video on inference: what happens if the model goes up 30%, and what happens if real inference (when the list price stops being this soft) is double. If the case only works on today's rate, you do not have a case, you have a promotion.

What does not help is using $/1M tokens as a product KPI (key performance indicator). It helps you negotiate with the provider. It does not help you approve an agent in a SaaS.

A worked example, step by step

This is an illustrative example, not a customer: a sum you can repeat with your rates.

Imagine a support agent that drafts a reply for a ticket and does not send it to the customer. As a sample rate, in September 2026, a mid-tier model on the public tables is on the order of $2 per million input tokens and $12 per million output tokens. The exact price changes every quarter; the method of the sum does not.

1. Initial context. Ticket plus three help articles: 8,000 input tokens. 8,000 / 1,000,000 x 2 = $0.016.

2. Two tool calls. Search the docs and read the order. The agent resends context. Another 12,000 input tokens and 600 output. Input: $0.024. Output: $0.007.

3. Retry. The docs tool returns too much. The agent asks again with more context: 15,000 input tokens and 400 output. Input: $0.030. Output: $0.005.

4. Final draft. 500 output tokens. $0.006.

Tokens for this run: about $0.09. It looks cheap, and that is exactly the number on the vendor slide.

5. Evaluation. One in 20 tickets goes through an automatic judge (about $0.02). Amortized: $0.001.

6. Human. Reviews for 90 seconds. If that person's loaded minute is worth 0.80 EUR (about 48 EUR/hour), that is 1.20 EUR.

Cost of the accepted run: about $0.09 of model plus 1.20 EUR of human. The token is a fraction. At 10,000 tickets a month, the model is about $900, and the human, if they review every one, runs past 12,000 EUR. That is why the token price does not decide margin: what decides it is how many runs come out right without repair.

If 5% enter a loop (eight tools, 40,000 tokens of context, a reasoning model), that 5% can match or beat the spend of the other 95%. Budget the tail, not only the happy path.

Now the stress test: multiply inference by two. If the case still works, you have margin. If the margin disappears, that flow should not be autonomous yet. It fits what we already covered in production: a demo is not an agent.

What to optimize first

Do not start with the cheapest model: start with what multiplies tokens.

Cut tools and context. Every document "just in case" is paid on every step, and every extra tool is a chance to dump junk and retry. The cheapest agent is the one that does one thing.

Put a stopping rule in place. A cost limit per thread, per tenant, and per day, a max number of tool calls, and a kill switch. Without that, the p95 has no limit.

Route. The large model is for the 10% of hard turns, not for greeting or classifying. People default to the most expensive model; the real saving is reserving expensive reasoning for when it changes the outcome.

Measure so you can drop down. Without evaluation you do not dare change models, so you pay the premium forever or you drop blind and pay in rework.

Remove the human where the risk allows it. Review minutes are, in many flows, the big line. Leaving them on the send-to-customer step and removing them from an internal summary is not "less quality": it is unit economics. BCG puts it in Radar 2026: you have to track tangible outcomes. ROI (return on investment) is not a token projection.

When a cheap model gets expensive

A cheap model gets expensive when the failure costs more than the token you saved.

If the hit rate drops and a human rewrites, you traded $0.04 for two euros of someone who was not in the plan. If the model calls tools badly and dumps 20,000 tokens of noise, the "saving" is eaten by the next step. Without evaluation you do not know which of the two is happening.

An expensive model used for everything is expensive too. Classifying a ticket with a frontier model is a luxury; extended reasoning only pays for itself on hard tasks.

The question is not "which model is cheapest?", but "which is the cheapest that keeps the success rate and the p95 your margin can stand?". If you cannot answer that this week, you do not have a provider problem: you have a measurement problem.

There are cases where the agent does not pay: low volume, small value per task, high human repair, or a flow with no source of truth. You do not need a cheaper model there; you need a different automation, or you wait. Alberto's framework is this: if it pays by a lot, run it; if it pays by a little, optimize; if it does not pay even with stressed inference, put it on ice.

Checklist: cost per run, not the token price

Before you approve the agent (or fight the vendor rate), the team should be able to answer yes to this.

Do you know the cost of a correct run this week, not just the invoice total?

Do you split model, tools, retries, and human minutes?

Do you look at the p95 and the fat tenant, not only the average?

Is there a cost limit per thread and per tenant, with an owner?

Does the case survive a 30% model increase, or double inference?

Do you have enough evaluation to drop models without guessing?

If two or more are missing, negotiating the price per million does not change the margin: it changes the story.

Once you have those lines, the rest is arithmetic. At Devic that account runs in the harness: observability and cost control per run, with plans to measure volume before you commit the margin. It is not a universal price: it is how you see costs scale before you turn the agent on for every tenant.

The list price of a token may keep falling, or it may not. OpenAI, Anthropic, and Google may be charging below the real cost of inference today, and that is not a floor you can build margin on. What does not fall on its own is the agent invoice, unless you measure each run and protect yourself with a cap and a stress test.