LLM cost optimization in Chicago.

The technical reality of document-pipeline spend in Chicago.

Questions from Chicago buyers.

What should processing a document actually cost, and how do we know if we are overpaying?

For the extraction and classification work Chicago operations run (rate confirmations, claims pages, invoices), a well-engineered pipeline lands between a tenth of a cent and two cents per document depending on length and field count; pipelines built quickly during the pilot era often run five to twenty times that. The overpayment signatures are recognizable on sight: a frontier model doing single-label classification, full documents sent where two pages carry the fields, prompts re-explaining the schema on every call instead of caching it, and synchronous processing at 2 a.m. on documents nobody reads until 8. The audit takes a month of your actual volume, computes true cost per document by type, and benchmarks it against what the right-sized pipeline would pay. The gap, at tens of thousands of documents a month, is usually a number your controller wants in writing.

Our document runs happen overnight. How much does batch processing actually save?

Half, before the other optimizations stack. The major providers price batch APIs at roughly 50 percent of synchronous rates in exchange for a completion window (typically under 24 hours, usually much faster) that overnight workloads never notice. The engineering is modest: a queue that collects the day's documents, a submission job, polling or webhook collection, and retry handling for the stragglers, all of which we build as standard plumbing. Batch then multiplies with the other levers: a workload moved to a right-sized model (say, 8x cheaper), with cached schema prompts (input costs down 60 percent), submitted in batch (half price again) compounds into the order-of-magnitude reductions that sound invented until the invoice confirms them. If your documents queue overnight and you are paying real-time prices, that is the first fix and the easiest one.

Does adding human review queues for exceptions raise or lower total cost?

It lowers it, which surprises teams who see review labor as the expense being eliminated. The mechanism: without an exception queue, pipelines get tuned for the worst documents (bigger models, longer prompts, multiple passes) so that nothing fails silently, and every clean document pays that paranoia tax. With confidence thresholds and a review lane, the pipeline runs lean for the 90-plus percent of documents that are routine, and the genuinely ambiguous minority gets human eyes that cost less than the across-the-board model upgrade did. The accounting that matters is fully loaded cost per correctly processed document, including review minutes and error-correction downstream; we compute exactly that in the audit. Operations that embraced exception lanes typically cut model spend 30 to 50 percent while error escapes drop, because doubt is routed instead of papered over with tokens.

How do we measure savings credibly, so finance trusts the number?

Baseline first, change second, and never let the two blur. The method we use: a month of pre-change usage instrumented at the request level (tokens, model, feature, document type), converted to cost per unit of work (per document, per call, per claim), which becomes the signed baseline. Changes then land sequentially with the same instrumentation running, so each fix's contribution is attributable: caching moved input cost X percent, routing moved per-document cost Y, batch moved the overnight run Z. The deliverable to finance is a before-and-after per-unit table on identical work mix, plus the dashboards that keep computing it monthly, because savings that cannot be re-verified next quarter decay into folklore. We also flag volume effects honestly: if document count doubles, the bill can rise while per-unit cost falls, and the report keeps those stories separate.