LLM cost optimization in San Diego.

The technical reality of research-scale spend in San Diego.

Questions from San Diego buyers.

Our lab is debating buying GPUs versus staying on APIs. What does the honest math look like?

It looks like a utilization spreadsheet, and labs usually lose the GPU case to it. The ledger: a serving-capable GPU setup (purchased or rented), power, and the postdoc-hours that keep inference infrastructure healthy, divided by tokens actually served, against API pricing that includes batch and caching discounts and falls every few quarters. Research workloads are bursty by nature (intense during analysis pushes, idle between), which wrecks utilization: a box busy 15 percent of the time costs multiples of the API path per token. Where the GPU case genuinely wins in San Diego: data governed by policies that bar external inference even in-tenancy (some IRB protocols, some sponsor agreements), sustained pipelines (a tuned small model processing instrument output around the clock), and grant structures where capital spends easier than recurring costs, which is a real consideration we price rather than mock. The audit runs your actual volumes and policies through both columns and the recommendation ships with the spreadsheet.

Can we track AI costs per experiment or per project, the way we track other research costs?

Yes, and for grant-funded work you arguably must. The plumbing: every model call tagged with project and experiment identifiers at the call site (a one-line discipline in shared lab tooling), aggregated into per-project dashboards that map cleanly to grant accounting. The payoffs compound: per-experiment costs make methods sections and data-management plans honest ('analysis used approximately $X of inference'), grant budgets get real numbers instead of guesses for the next proposal, cost anomalies surface attributed (the pipeline that quietly tripled, the student script in a retry loop), and shared-resource fairness becomes visible when one project consumes the lab's allocation. For core facilities charging back to PIs, the same tagging becomes a billing system. We install the tagging pattern and dashboards as standard in research engagements, because untagged spend in a multi-project lab is unattributable by definition, and unattributable spend gets argued about instead of managed.

Our embedding and document-processing pipelines run constantly. Where does that spend hide?

In re-embedding, over-chunking, and model mismatch, three quiet leaks. Re-embedding: pipelines that re-process unchanged documents on every run (no content hashing) pay full price for idempotent work; a hash-and-skip layer typically eliminates 60 to 90 percent of embedding calls in mature corpora. Over-chunking: aggressive overlap and tiny chunks multiply token counts beyond what retrieval quality needs; we tune chunk size and overlap against your actual retrieval benchmarks rather than tutorial defaults, and the corpus often shrinks by a third in token terms. Model mismatch: embedding models priced for quality the task does not need, or frontier models summarizing where small models match (provable on your eval set). Batch endpoints take the remaining volume at half price, since corpus processing is never latency-bound. Research pipelines that have run unexamined for a year routinely cut 50 to 70 percent here, which in grant terms is a budget line returned to actual science.

Part of our work is export-controlled. Do any cost levers survive ITAR constraints?

Most of them, applied inside the boundary. ITAR-relevant inference runs on GovCloud-class or certified self-hosted infrastructure with US-person access controls, and within that perimeter the standard toolkit still operates: prompt and glossary hygiene (tokens are tokens everywhere), caching where the serving stack supports prefix reuse (self-hosted stacks like vLLM support it natively), right-sizing across whatever model lineup your environment certifies, and batch scheduling on your own queue, where the discount is utilization smoothing rather than a provider rate card. The lever that changes most is model choice: the certified lineup is narrower, so benchmarking within it matters more, and the quality-cost frontier sits differently than on commercial APIs. What never works is routing controlled work through commercial endpoints to save money; the audit treats the boundary as fixed and optimizes the inside, and the report keeps controlled and uncontrolled workloads as separate ledgers, which your compliance officer will want anyway.