57% lower model cost in an offline replay

One answer included several paid calls

An options answer could take several model steps. Each step read context from earlier steps and tool results. The trace review found the same large input being billed again within a turn.

An options-chain result also held duplicate data and unused fields. The full app payload was reaching the model. A better prompt alone would still leave that source of cost in place.

For the same check on another system, follow the guide to tracing growing agent costs.

We changed two boundaries

We changed what reached the next model call. Other parts of the app kept the full tool data.

• Cache reuse within a turn. We added rolling cache breakpoints to the model adapter. Later calls could reuse a shared prefix. The code capped the marker count. It repaired requests that went over the cap.
• Smaller tool results. We built a trimmed copy for the model. It dropped duplicate raw data and unused source fields. It rounded numbers and narrowed the expiry window. Other app callers kept the full payload.
• Contract checks. Tests checked cache markers and rules for the trimmed data. If that step failed, the app could use the full payload and report the error.

How the replay was checked

The team captured a baseline of 11 production options turns. The shipped code replayed the data changes and computed the costs. This used captured work. It was not a controlled before-and-after on new live traffic.

The checks measured the size of the options payloads. They checked that other tools passed through unchanged. Code tests checked cache use and the data rules. Those tests do not prove that each final answer would be just as useful.

Before a wider rollout, use a separate cost-and-quality comparison with acceptance rules and live observation.

What the record shows

An earlier cost model projected a different result on one turn. That is not the headline here. We use the narrower replay result from the record.

What this result does not establish

• A live saving. No live before-and-after was recorded. We do not claim this replay caused a lower monthly bill.
• Equal answer quality. Smaller payloads and passing code tests are useful checks. The final answers still need their own review.
• A result for every task. This change targeted options-heavy turns. It does not forecast savings for short, single-call tasks. It is not an average for the whole service.
• A saving for another company. Its context, tools, models and cache use need their own baseline.

What to try on your own agent

Start with one costly task. Inspect what each model step reads. Split the bill into uncached input, cache reads, cache writes and output. Look at the tool data sent to the model. The app screen alone may not show it.

Test a smaller payload or better cache use on the same saved tasks. Keep needed fields and sources. Compare useful results, full-run cost, time and human review. Then test the change on live work.

Use the free agent cost calculator to organize your starting numbers and download a worksheet.