LLM cost optimization in Houston.

The technical reality of constrained-deployment spend in Houston.

Questions from Houston buyers.

Our inference is locked to in-tenancy deployment. Which cost levers survive that constraint?

All the ones that matter. In-tenancy inference through Bedrock or Azure OpenAI carries the full toolkit: multiple model sizes for difficulty-based routing under the same governance, prompt caching with the standard steep discounts, batch endpoints at roughly half price, and commitment pricing (provisioned throughput) where sustained volume justifies it. The audit treats your tenancy as the field of play rather than a handicap: we benchmark the eligible model lineup on your documents, restructure prompts for cache stability, and move the overnight report runs to batch, all without a single byte leaving your account or a security review reopening. What the constraint does rule out is consumer-endpoint arbitrage and external gateway services, neither of which belongs in a Houston operational stack anyway. The savings numbers in our reports here are all achievable inside the boundary your policy already drew.

We bought GPUs for self-hosted models. How do we know if they are earning their keep?

Utilization math against the API alternative, run honestly. The ledger: your GPU costs (amortized hardware or monthly rental, power, the engineering hours that keep serving infrastructure healthy) divided by actual tokens served gives a true per-token cost; compare that against current API pricing for equivalent-quality models, including the batch and caching discounts you would collect there. The common Houston finding: GPUs bought for data-sovereignty reasons run at 10 to 20 percent utilization, making their effective per-token cost multiples of the API path, while the sovereignty rationale would be equally satisfied by in-tenancy cloud inference. Where self-hosting genuinely wins: sustained high-utilization narrow workloads (a tuned small model processing field reports around the clock), true air-gap requirements, and edge deployment near operations. The audit computes your actual utilization and prices both worlds, and if the recommendation is to repurpose the GPUs, it comes with the migration plan and the spreadsheet that justifies it.

Daily field reports arrive overnight. What does moving them to batch processing save?

Half, immediately, and the work is plumbing rather than redesign. Batch endpoints price at roughly 50 percent of synchronous rates in exchange for a completion window measured in hours, which overnight report cycles never notice: reports landing between 6 p.m. and 4 a.m. get submitted as a batch, results return before the 6 a.m. rollup, and nobody downstream can tell the difference except finance. The savings stack from there: the extraction-then-summarize pattern (structured fields first via a small model, narrative on top) usually right-sizes the model mix at the same time, and cached glossaries and schemas cut the input bill further. The plumbing we install is standard: a collection queue, batch submission, webhook or polling collection, retry handling for stragglers, and a fallback to synchronous for the rare urgent report. For operations processing thousands of reports nightly, this is the cleanest 50-plus percent in the portfolio.

Do domain glossaries make prompts expensive? Ours are getting huge.

Only when they are unmanaged; structured properly, they are nearly free. The failure mode: glossaries grow by accretion (every term anyone ever requested), get pasted into every prompt, and bill full price on every call. The fixes, in order: prune against usage (we log which glossary entries the model actually needed across a month of traffic; typically a third of entries do all the work, and the rest are dead weight), scope by document type (a drilling-report prompt does not need pipeline-integrity vocabulary; per-workflow glossary subsets cut tokens immediately), and cache the survivors (a pruned, scoped glossary as a stable prefix bills at the 90-percent cached discount after the first call). The result is glossaries that cost single-digit percent of the input bill instead of half of it, with retrieval quality unchanged or improved, since smaller relevant glossaries also reduce the model's distraction. Glossary hygiene is one of the quiet wins specific to technical-domain shops like Houston's.