Token cost optimizationThesis Finance
Reducing AI costs in an investing assistant
- Problem
- Thesis Finance uses AI to answer questions about portfolios and stock options. Some answers required several model calls that repeatedly charged for the same background information.
- Solution
- We added caching so the model could reuse that information at a lower cost, and removed duplicate and unnecessary data from the options results sent to it.
- Outcomes
- 57% lower model cost in an offline replay of 11 recorded options requests: $2.09 to $0.90 per answer.
- Options data sent to the model was 2.5–6× smaller. The savings have not been measured on live traffic.