Product note
Reducing LLM cost in Apex
A product lesson from cutting Apex model cost by roughly 95%.
The first version gave an expensive model too much responsibility. It received broad market context, performed much of the reasoning, and generated the explanation. The result could look intelligent while remaining expensive and difficult to reproduce.
The better architecture separated computation from communication. Python services now calculate market state, levels, conflicts, and risk. The LLM receives a compact evidence package and translates it into a consistent decision format.
The change also made the system easier to test, trace, guard, and price. I now use an LLM for explanation and language tasks, while deterministic services handle calculations that need to be repeatable.