Should I trim context or switch to a cheaper model?
Choose a cost experiment from the evidence: repeated input, large tool results, expensive output or extra retries.
By OlypsisUpdated 4 minute read
Overview
Start with the part of the task that costs the most. Does the agent keep reading the same text? Check context and cache use. Does it spend more on output? Test a different model or a smaller task. Compare the cost of useful results before keeping a change.
Make the baseline comparable
Choose tasks with clear pass rules. Group them by the job they do. Record model spend, tool fees, time to finish, retries and review effort. Keep short lookups apart from long research tasks.
A model with lower token prices can still cost more per completed task. It may write more text, make more calls, or need retries and fallbacks. Count that work and the time people spend reviewing or fixing it.
The agent cost calculator helps separate token categories and relate spend to accepted results. Its scenario is an estimate to test, not a promised saving.
Choose the first test from the trace
| Evidence | Try first | Watch for |
|---|---|---|
| Large duplicated tool output | A smaller model-facing payload | Missing fields, units or source evidence |
| Repeated stable input at full rate | Cache reuse and prompt ordering | Write charges, misses and expiry |
| Long output or reasoning | A narrower answer or a different model | Lost reasoning steps or incomplete results |
| Repeated calls and retries | Tool contracts and stopping conditions | Tasks ending before the work is done |
| Tool charges dominate | Fewer redundant external lookups | Stale data or reduced coverage |
Test context changes at the boundary
Keep the full tool output. Make a smaller copy for the model. Start with duplicate data and fields the task does not use. Check needed values, units and IDs. Then replay the task.
Keep the rules that make the agent safe and useful. If a source is too long, try a more focused search or a smaller task. Do not just cut off the end. Keep the facts the agent needs to support its answer.
Our token optimization checklist walks through prompts, tool output, caching and model choice.
Test model changes on the same work
Test a new model with the same tools, task inputs and pass rules. Save failed runs as well as good ones. Look for more tool calls, longer answers, missed steps and extra review.
A model may suit one step but struggle with another. If you route tasks to different models, count the router’s cost and errors too. Keep a way to send a failed task back to the current model.
Try one bounded extraction or classification step
Choose a job with a result you can check. For extraction, that could be the date and total on an invoice. For classification, it could be assigning a support ticket to a known category. Test a less expensive candidate on the same saved examples as the current model.
- Ask for only the fields or category the next step needs. Validate the format and the values against your source records.
- Include hard and ambiguous cases. Check when the model should return an unknown result or ask for review. Do not assume it will catch all its own mistakes.
- Set a retry budget before the test. For example, allow one attempt to repair invalid output, then use the current model or a person. Stop when that path also cannot finish the task.
- Count every call, billed token, retry and fallback, plus review and correction time. A short visible answer does not tell you the full billed usage.
- Record both automatic completion and final correct completion. Count each task once, even when it needed several attempts.
See how exception handling changes the bill in the worked 70/30 cost example.
Keep the change that survives the comparison
Test each change on its own. That shows which one helped and which one caused a failure. Then test them together. Their joint saving may differ from the sum of each saving.
Set the quality, time and review limits before seeing the results. Keep the cheaper option only if it meets them. Check it again on live work after release. Keep a way to undo it.
Use this guide to testing a cheaper agent to write the acceptance rules and rollout checks.
Have a project in mind? Tell us what you’d like to improve, and we’ll help you work out where to start.
Book a call