The monthly inference invoice usually lands with the cloud bill and gets treated the same way: an infrastructure cost, owned by an infrastructure function, reviewed when it grows uncomfortable.
That framing costs money. The cost profile of an agentic system is not an operational artifact. It is a direct readout of architectural decisions, and it is frequently more honest than the architecture documentation, because nobody updated the diagram after the third refactor but the bill reflects what the system actually does.
Five things the bill tells you
Cache hit rate reveals prompt assembly discipline. Prompt caching depends on a stable prefix. Anything that varies at the top of the prompt — an injected timestamp, a session identifier, retrieval results in non-deterministic order — invalidates it. Teams routinely ship this without noticing, because functionally nothing is wrong. The fix is usually an afternoon of reordering prompt construction, and the effect on the bill is disproportionate to the effort.
Rising tokens per task reveals context accumulation. Conversation and agent state grow without a summarization or eviction strategy. Cost rises faster than volume, latency degrades alongside it, and quality often degrades too as relevant material gets buried in accumulated history. If cost per task is trending upward while task complexity is flat, this is nearly always the cause.
Fan-out ratio reveals orchestration design. One user request becoming twenty model calls is sometimes exactly right and sometimes evidence that the problem was decomposed further than it warranted. The ratio is worth knowing deliberately rather than discovering. Multi-agent architectures are frequently adopted for their conceptual elegance and paid for at a multiple.
Retry share reveals failure containment. Retries against non-deterministic failure amplify rather than resolve. A malformed tool call retried three times at full context costs four times the original call and still fails. Retry policy is a cost decision as much as a reliability decision, and the two are usually set by different people.
Model mix reveals routing discipline. Frontier models performing classification, extraction, and routing work that smaller models handle competently is, in our experience, the largest single cost lever available on most deployed systems and among the least frequently pulled. Tiering is not a downgrade — it is matching capability to task.
The costs that are not on the model invoice
Focusing only on the inference line understates the picture, sometimes substantially.
Evaluation runs consume inference. A team with genuine evaluation discipline — running a golden set against every prompt change and model version — is generating real spend to do so, and that spend is correct, but it needs to be budgeted rather than discovered. Embedding generation and periodic reindexing of a large corpus carry their own recurring cost. Observability at agent granularity produces log and trace volume well beyond what a conventional application generates, and storage and query costs follow. Where a human review step sits in the loop, that labor is frequently the largest cost in the transaction and rarely appears in the same report as the model spend.
The consequence is that cost per transaction, calculated honestly, is often materially higher than the inference figure that gets quoted internally. On fixed-price work, the difference between the quoted figure and the honest one is margin that was committed before anyone measured it.
Attribution before optimization
None of this is manageable without attribution. A single aggregate monthly number tells you the system costs too much and nothing about where.
Instrument by client, by feature, and by transaction type from the first week of build. This is inexpensive when it is designed in and awkward when it is retrofitted, and it is the difference between an optimization effort that targets the actual driver and one that spends a sprint tuning something responsible for four percent of spend. It also matters commercially — on a multi-client platform, per-client attribution is what allows you to price the next engagement from evidence rather than from an estimate.
Why this belongs on a partner's agenda and not only an engineer's
Under time and materials, inference cost is largely the client's exposure. Under fixed price, outcome-based pricing, or a managed service arrangement, it is yours.
That creates a structure worth stating plainly: it is the one significant input cost that scales with client success. The more the client adopts the system you built, the more it costs you to operate. Absent volume assumptions written into the contract with a defined mechanism for what happens when they are exceeded, the engagement becomes less profitable precisely as it becomes more successful.
That is not a technical problem. It is a commercial structure problem, and it is set at contract signature by people who typically have not seen the cost model.
Making it a design constraint
Cost per transaction belongs in the architecture review as a target, not in the month-four operations review as a finding. Treat it the way a competent team treats a latency budget: state the number, instrument against it early, and require a justification when a design decision spends against it.
Five questions surface most of the exposure in a design review:
What is cost per transaction at expected production volume, and how was that estimated?
What does that number become at ten times volume?
What is the cache hit rate, and what invalidates the prefix?
What is the retry policy, and what does a failed call cost when it is retried?
For each model call in the path, why is that model on that call?
Teams that can answer those five have usually built something operable. Teams that cannot are typically two to three months from an uncomfortable conversation about the business case.