What an AI agent actually costs in production
Cost
The first invoice surprises most teams. Not because the price per Text split into the small pieces a model computes on. An English word is typically one to two tokens. Model pricing is always quoted per million tokens.is high, but because the intuition is wrong: a SaaS product costs more when more people use it, an agent costs more every time it thinks one more round.
The cost is in the context, not in the answer
An A model allowed to call tools and run several turns on its own, rather than answering once and stopping. is a loop: task, tool call, result, reconsider. The surprising part is that every turn resends the entire Everything sent to the model in a single call: instructions, conversation history and tool responses. The model remembers nothing between calls, so the context is resent every time.. The model remembers nothing between calls, so the The fixed instruction at the front of the context that tells the model what to do. It is identical on every single call., the tool definitions and all prior history go along each time.
| Turn | Fixed context | History | Sent in |
|---|---|---|---|
| 1 | 3,000 | 500 | 3,500 |
| 2 | 3,000 | 1,200 | 4,200 |
| 3 | 3,000 | 2,000 | 5,000 |
| 4 | 3,000 | 2,800 | 5,800 |
| Total | 12,000 | 6,500 | 18,500 |
One request, 4 turns, tokens per turn. Note the total row: 12,000 of the 18,500 tokens are the same system prompt sent 4 times.
So an agent taking 6 turns does not cost 6 times one turn. It costs the sum of a growing context, dominated by tokens going in rather than the answer coming out.
Run the numbers
Put your own figures in, the structure holds. A support tool with 20,000 requests a month, 4 turns each, and a fixed 3,000 token system prompt.
Roughly 18,500 tokens in per request against 1,200 out. Scaled up: 370 million tokens in per month, of which the fixed 3,000 alone account for 240 million. Close to two thirds of the bill, before the agent has done anything useful.
Close to two thirds of the bill was the same system prompt, sent again.
What that is in money
370 million tokens in and 24 million out per month. Same calls, same system, same task. The only thing that changes down the table is which model you point at.
| Running everything on | In | Out | Per month |
|---|---|---|---|
| Frontier | $1,850 | $600 | $2,450 |
| Mid-tier | $740 | $288 | $1,028 |
| Small | $52 | $7 | $59 |
Prices retrieved 7 August 2026 for the three tiers described in the pricing article. The difference between the top and bottom row is 41 times, on a system doing exactly the same work.
Same system, same task, same number of calls. $2,450 or $59.
The point is not that you should run everything on the bottom row. It is that nobody should run everything on one row. Put the trivial steps at the bottom and keep the frontier where it is genuinely needed, and the bill lands near the floor while quality stays at the ceiling where it matters.
The spread between models is wider than people think
A review of frontier pricing in July 2026 found a 643x gap between the most and least expensive price per million output tokens, across models you call with nearly identical requests. Same task, same arithmetic above, and an entirely different final number depending on what you point it at. We have written more about how the prices move in The price of AI is collapsing.
The large model is rarely the right default. An agent loop has a few demanding steps and many trivial ones, and the trivial ones are most of the volume.
4 things that work
Pick the model per step, not per system
The routing does not need to be clever, just a table from step type to model size. The gain comes from moving the volume off the expensive model.
| Step in the loop | Share of volume | Difficulty | Model |
|---|---|---|---|
| Classify the request | High | Low | Small |
| Extract a field | High | Low | Small |
| Summarise a tool response | Medium | Medium | Medium |
| Plan the next step | Low | High | Large |
| Write the reply | Medium | Medium | Medium |
Note where the large model sits: on the single step with low volume. That is where it earns its price.
The provider stores the part of the context that does not change, so you do not pay full price for it on every call. what does not change
The system prompt is identical on every call and sits first in the context. Most providers offer caching for exactly that part. In the arithmetic above it is the single line item that matters most.
Cap the history
An agent left to run freely will sometimes spend 20 turns on something it cannot do. A hard cap on turns cuts the tail where cost runs away.
Measure cost per run, not per month
A monthly total tells you nothing about where the money goes. Log tokens and model per step, aggregate by task type, and you will see which 5 percent of requests account for half the bill.
This connects to evaluation
All of it depends on answering one question: did it get worse? Switching model without being able to measure quality is not cost control, it is guessing.