LLM cost optimization in Seattle.

The technical reality of platform-scale spend in Seattle.

Questions from Seattle buyers.

When does Bedrock provisioned throughput beat on-demand pricing?

When utilization is genuinely sustained, and the breakeven deserves arithmetic rather than instinct. Provisioned throughput buys committed model units at a fixed rate; it wins when your traffic keeps those units busy most of the day, and it loses (sometimes badly) when bought for peak load that occurs four hours a day. The audit pattern: instrument a month of traffic by model and hour, compute the utilization curve, and price three scenarios (pure on-demand, provisioned base with on-demand burst, full provisioned). Most Seattle platforms land on the hybrid: a provisioned floor sized to the steady tier, on-demand absorbing spikes, and batch absorbing everything that is not latency-sensitive. The same review checks whether you are on current model generations, since newer models inside Bedrock frequently deliver equal quality at lower per-token rates, and version inertia is its own quiet cost.

Can routing and caching work entirely inside our cloud tenancy?

Yes, fully. In-tenancy inference through Bedrock (or Azure OpenAI) supports the complete optimization toolkit: multiple model sizes under the same governance umbrella for difficulty-based routing, prompt caching with the standard steep input discounts, and batch endpoints at half price, all without traffic leaving your account or your security review reopening. The routing layer itself is your code, deployed in your tenancy, logging model-and-reason per request into your existing observability stack. This matters in Seattle because the alternative pitch (route through an external gateway service) often trades cost savings for a new vendor review and a new data path, which on enterprise platforms costs more in calendar time than it saves in tokens. We build the routing and caching inside the boundary you already govern, and the security packet does not change.

How does LLM spend fold into our existing FinOps practice?

The same way compute did: tagged, allocated, budgeted, and anomaly-monitored, with LLM-specific dimensions added. Concretely: every model call carries cost-allocation tags (team, product, feature, environment) flowing into Cost Explorer and your existing FinOps tooling; per-token costs roll up beside per-request and per-feature views, since finance thinks in features and engineers think in tokens; budgets and alerts ride the same channels as your other cloud spend (a model-spend anomaly should page like a compute anomaly); and unit economics (cost per request, per tenant, per workflow) get computed continuously rather than at invoice shock. The work is mostly plumbing discipline: consistent tagging at the call site, a small aggregation layer, and dashboards in the tools your FinOps team already lives in. LLM spend that lives outside the FinOps frame stays a mystery line item; inside it, it becomes just another managed resource.

What does good cost observability look like for a platform team running many AI features?

Four layers, each answering a different audience. Per-request telemetry at the base: tokens in and out, model, latency, cache hit, feature tag, sampled prompts for debugging, retained briefly. Per-feature economics above it: cost per request and per active user by feature, trended, so product owners see their own line. Allocation views for finance: spend by team and product with month-over-month attribution, in the FinOps tooling they already use. And drift alarms across it all: cost-per-request regressions (a prompt change that doubled tokens), cache hit-rate drops, routing distribution shifts, and volume anomalies, paged to the owning team rather than discovered at invoice time. Platforms running dozens of AI features cannot manage what aggregates into one number; the observability build is usually the first deliverable here because every later decision (routing, caching, pricing) reads from it.