"Which LLM should we use?" is the wrong first question. The right first question is: "What task are we trying to automate, what accuracy do we need, and what can we spend per request?"
Once you have answers to those three, the model choice usually becomes obvious. This guide covers the six models most commonly used in production business systems and maps them to the task types where they actually perform well.
The model landscape in 2026
Six models account for the vast majority of production business deployments:
| Model | Provider | Context window | Self-host option | Key strength | |---|---|---|---|---| | GPT-5 | OpenAI | 400K | No (Azure hosted) | Broad capability, tool use reliability | | Claude Opus 4.8 | Anthropic | 1M | No (AWS/GCP hosted) | Frontier reasoning, coding, long-horizon agents | | Claude Sonnet 5 | Anthropic | 1M | No (AWS/GCP hosted) | Near-Opus quality at lower cost, instruction following | | Claude Haiku 4.5 | Anthropic | 200K | No (AWS/GCP hosted) | Fast, cheap classification and extraction | | Gemini 2.5 Pro | Google | 1M | Vertex AI managed | Very long documents, multimodal | | Llama 4 | Meta | Up to 1M | Yes (self-hosted) | On-premises, data residency requirements |
Cost comparison at production scale
Costs are input/output per 1M tokens as of mid-2026. These change regularly — verify current pricing before budgeting.
| Model | Input (per 1M tokens) | Output (per 1M tokens) | |---|---|---| | GPT-5 | $1.25 | $10.00 | | Claude Opus 4.8 | $5.00 | $25.00 | | Claude Sonnet 5 | $3.00 | $15.00 | | GPT-5 mini | $0.25 | $2.00 | | Claude Haiku 4.5 | $1.00 | $5.00 | | Gemini 2.5 Pro | $1.25 | $10.00 | | Llama 4 (self-hosted) | Infra cost only | Infra cost only |
What this looks like at scale
A customer support chatbot processing 1,000 conversations/day at 2,000 tokens per conversation — assume a 50/50 input/output split — works out to ~30M input + 30M output tokens per month.
Approximate monthly cost by model:
- GPT-5: ~$340/month
- Claude Opus 4.8: ~$900/month
- Claude Sonnet 5: ~$540/month
- GPT-5 mini: ~$68/month
- Claude Haiku 4.5: ~$180/month
- Llama 4 self-hosted: ~$200–$400/month infra (depending on GPU cloud pricing)
For high-volume, lower-complexity tasks, moving from a flagship model to its small tier (GPT-5 → GPT-5 mini, or Sonnet 5 → Haiku 4.5) is roughly a 5x cost reduction. The accuracy difference for most classification and extraction tasks is minimal.
Recommendations by use case
Complex reasoning and multi-step agents
Best choice: Claude Opus 4.8 or Claude Sonnet 5, GPT-5 competitive
By mid-2026 Claude leads on long-horizon agentic work — fewer malformed tool calls over long runs, strong instruction following when the system prompt is complex. Opus 4.8 is the frontier option; Sonnet 5 gets you most of the way at lower cost. GPT-5 is a strong alternative and worth including in your eval.
Test all three on your actual tasks before committing. The quality difference for reasoning tasks is often smaller than marketing materials suggest.
Long-document processing (contracts, reports, technical specs)
Best choice: Claude Sonnet 5 or Gemini 2.5 Pro
Claude Sonnet 5 now reaches a 1M-token context window, which covers all but the largest document sets. Gemini 2.5 Pro also reaches 1M and is a fine alternative, especially for multimodal documents. Whichever you pick, test reasoning quality at your actual context length — quality can degrade toward the top of any model's window.
High-volume extraction and classification
Best choice: GPT-5 mini or Claude Haiku 4.5
These tasks don't require frontier reasoning capability. Invoice field extraction, sentiment classification, intent routing, entity extraction — all of these work well at the small-model tier at a fraction of the cost. Route to the frontier model only when the smaller model fails.
A common pattern: use GPT-5 mini (or Claude Haiku 4.5) for 80% of requests, fall back to a frontier model for the 20% that fail the quality check. This gets you frontier accuracy on hard cases at a fraction of frontier pricing on the full volume.
Code generation and technical tasks
Best choice: Claude Opus 4.8 or Claude Sonnet 5
By mid-2026 Claude is the strong default for code generation, review, and agentic coding — Opus 4.8 at the frontier, Sonnet 5 for a cheaper workhorse. GPT-5 is competitive and worth benchmarking. Don't take anyone's benchmark on faith: run your own coding eval suite (real diffs, real review tasks) and pick on measured results, not leaderboards.
On-premises or strict data residency requirements
Best choice: Llama 4 (self-hosted)
If your data cannot leave your infrastructure — certain healthcare, government, or financial use cases — self-hosted Llama 4 is the practical option. The quality gap versus the frontier models has narrowed but is still real for the hardest reasoning; for many structured tasks (extraction, classification, summarization with good prompts) it's more than acceptable.
Self-hosting has real costs: GPU instances, model serving infrastructure, monitoring, and maintenance. Budget $3,000–$8,000/month for a production-ready self-hosted setup, depending on scale.
Compliance certifications
Compliance certification coverage varies significantly. Verify current status with each provider, as certifications change.
| Provider | HIPAA BAA | SOC 2 Type II | ISO 27001 | FedRAMP | |---|---|---|---|---| | OpenAI (Azure) | Yes (via Azure) | Yes | Yes | Yes (limited) | | Anthropic (AWS Bedrock) | Yes (via AWS) | Yes | Yes | In progress | | Google (Vertex AI) | Yes | Yes | Yes | Yes | | Self-hosted Llama | You own it | Your responsibility | Your responsibility | Possible |
For HIPAA-covered entities: using a frontier model through a cloud provider (Azure, AWS Bedrock, Vertex AI) typically provides a path to compliance via the cloud provider's BAA. Using the model provider's direct API (OpenAI's API directly, Anthropic's API directly) has different — and sometimes less clear — compliance coverage. Always verify the specific product covered under the BAA.
The model choice is not permanent
Teams over-optimize for model selection and under-optimize for system design. The model you launch with is not the model you'll run in 18 months — the landscape changes fast, pricing drops, and better options appear regularly.
Build your system with model abstraction: a configuration flag that lets you swap the underlying model without rewriting the application. A few hours of architecture investment upfront saves significant migration work later.
The practical advice: pick the model that performs well on your eval suite at a cost you can sustain. Run your evaluation suite, not benchmarks. What matters is performance on your tasks, not on MMLU. For most AI agent development work, Claude Opus 4.8 or Claude Sonnet 5 (or GPT-5) for complex reasoning and GPT-5 mini or Claude Haiku 4.5 for high-volume simpler tasks is the right starting point.