The pilot cost almost nothing. The rollout bill did not.

A pilot runs for a handful of users on a handful of documents. Cost is invisible. Then the workflow goes to the full team, volumes multiply, and the first monthly bill arrives. We have seen this at the end of many pilots, and the answer is rarely to switch the model off.

“Where is the money actually going?”

We start by attributing cost. Every model call is tagged with the workflow, the step and the model, so we can see cost per request and per step. The surprises are usually the same: very long contexts, retries, agent loops that call a tool again and again, and a large model doing work a small one could do.

“Does this task need the big model?”

Classification, extraction and routing are often fine on a smaller, cheaper model. Reasoning over conflicting sources may not be. We route by task and check the result against the evaluation set. If quality holds on the set, the cheaper route stays. If it does not, it goes back. The set makes this a measurement and not a guess.

“How much of this context is needed?”

Most of the tokens in a RAG agent are retrieved text. We tune how many passages are passed in, rerank so the useful ones come first, cache stable prompt prefixes, and reuse answers to repeated questions where it is safe to do so. Each change is tested for groundedness before it ships.

“What stops a loop from running all night?”

Agents need budgets. We set a maximum number of steps and tokens per task, limits per user and per workflow, and an alert when cost per task moves outside its normal range. A runaway loop then costs a few cents and a notification instead of an invoice.

“What does one finished task cost?”

Price per token tells a team little. We track cost per completed task and compare it with what the manual process costs. That number decides whether the workflow expands to the next team, and it keeps the conversation with finance concrete.