LLM cost optimization in Phoenix.

The technical reality of basis-point spend in Phoenix.

Questions from Phoenix buyers.

Everyone quotes 'fractions of a cent per item.' What does it actually take to get there?

Four specific engineering moves, applied in order. Right-sizing: classification, extraction, and template-grounded drafting move to small models (a tenth to a thirtieth of frontier pricing) wherever your labeled samples show parity, which for back-office document work is most of the traffic. Caching: the heavy stable content (schemas, templates, policy context) restructured into byte-identical prefixes billing at the 90-percent discount. Batch: everything that is not a human waiting (overnight queues, backlog runs) submitted at half price. And prompt hygiene: the accumulated instructions trimmed against an eval set, often 30 to 50 percent of tokens doing nothing. Stacked, those moves take a typical two-to-five-cent naive per-item cost to two to five tenths of a cent. The number is workload-specific, so the audit prices your actual items first, but the path there is the same four moves in the same order.

What does a prompt-bloat audit find, concretely?

Sediment, in predictable layers. Instructions that accumulated during development and never left: edge-case handling for situations that no longer occur, apologetic guardrails duplicated three ways, examples added during a bad week and kept out of superstition. Context that rides every call but earns nothing: full policy documents where two paragraphs apply, entire account histories where the open items suffice, schema definitions for fields the workflow stopped using. And structural waste: verbose JSON with long key names at volume, retry logic that re-sends maximal context, conversation history growing unbounded in long sessions. The method is empirical rather than aesthetic: we log a month of traffic, measure which prompt sections influence outputs on your eval set, delete what does not, and re-verify quality. Typical Phoenix back-office finding: 35 to 55 percent of input tokens removable with output quality flat, which at portfolio volume is the cheapest money in the engagement.

Which workloads should run on batch versus real-time pricing in a servicing shop?

Sort by who is waiting. Real-time pricing is for a human in the loop right now: an agent on a call needing a suggestion, a specialist reviewing a flagged item, a borrower in a chat. Everything else is a batch candidate, and in servicing shops that is most of the volume: overnight correspondence runs, document classification on the day's intake, QA scoring for tomorrow's review queue, backlog remediation projects, monthly statement narratives. The half-price batch discount is the headline, but the second-order win matters too: batch windows smooth your usage curve, which improves commitment-pricing math if you are on provisioned capacity. The plumbing is standard (collection queues, submission jobs, webhook returns, straggler retries, an urgent-path fallback), and we build it once per workload class. The audit's batch-eligibility pass usually reclassifies 60 to 80 percent of a servicing shop's tokens, which is why it comes right after the baseline.

Our controller wants to trust the dashboard, not the vendor. What does that take?

Reconciliation, attribution, and re-computability, the same properties any financial report needs. Reconciliation: the dashboard's monthly total ties to the provider invoice to the dollar, with the join logic documented, because the first time the numbers diverge the dashboard is dead to finance. Attribution: spend decomposes by workflow, document type, and department against the same dimensions the shop already budgets, so the AI line lands in existing cost centers rather than a new mystery bucket. Re-computability: unit costs (per letter, per file, per claim touch) recompute monthly from raw usage with the method documented, so this quarter's savings claim can be re-derived next quarter by someone who does not like us. We build the dashboards to those three properties and hand the keys to your controller's team, including the documentation. Optimization savings that finance cannot independently verify have a half-life of one budget cycle; verifiable ones compound into standing credibility.