Loading…
Stack Guide · OpenAI API
Getting your first GPT-5 response takes five minutes. Getting reliable structured outputs, predictable tool calling, and controlled costs at production volume takes considerably more care. The API surface area is wide and the documentation covers what things do, not how to use them well.
These are the patterns we've settled into across production builds: model selection, structured output approaches, tool calling, batch processing, and cost control.
Integrating the OpenAI API?
The cost difference between GPT-5 and GPT-5 mini is 16x. Whether that gap matters depends entirely on your task. Don't default to the most capable model, measure on your actual data.
GPT-5
~$2.50 / 1M input tokens
Complex reasoning, multi-step tasks, ambiguous instructions, cases where output quality directly affects user outcomes. The quality difference over GPT-5 mini is real for hard tasks, and negligible for simple classification or extraction.
GPT-5 mini
~$0.15 / 1M input tokens
High-volume classification, simple extraction, routing decisions, tasks where you can validate the output reliably. At 16x lower cost than GPT-5, the economics change dramatically. Run both on a sample of your actual prompts and measure, don't guess.
o3-mini
~$1.10 / 1M input tokens
Tasks that require extended reasoning: math, code generation, multi-step logic problems. The o3 models think before responding, which increases latency (5–30 seconds) but improves performance on reasoning-heavy tasks where GPT-5 makes mistakes.
Getting reliable JSON from the model is a solved problem. If you use the right tool. Don't reach for the other approaches.
Native structured outputs
Pass a JSON Schema to the `response_format` parameter with `type: json_schema`. OpenAI guarantees the response matches your schema, no parsing errors, no hallucinated fields. Available on GPT-5 and GPT-5 mini. This is the right approach for any production system that needs reliable JSON.
Function calling / tool calling
Define a tool with an input schema and instruct the model to call it. Slightly more flexible than native structured outputs for cases where you want the model to decide whether to return structured data or plain text. Works on all GPT-4 models.
Prompt-only JSON
Asking the model to 'respond in JSON' in the system prompt. Unreliable. The model will sometimes add prose before or after the JSON block, include comments, or produce malformed JSON under certain inputs. Don't use this in production.
Tool calling works reliably when you design the tool definitions carefully. Most problems trace back to one of these four issues.
Defining too many tools
If you define 15 tools in a single call, the model has to choose between 15 options. Performance degrades, the model sometimes picks the wrong tool or fails to call any tool. Keep tool sets focused: 3–7 tools per call is a practical limit. If you have many tools, route to the right subset based on context before calling the model.
Unclear tool descriptions
The model reads your tool name and description to decide which tool to call and what arguments to pass. Vague names like `process_data` or descriptions like 'processes the data' produce unpredictable behavior. Write tool descriptions the way you'd write documentation for a junior developer: be specific about what the tool does, what it returns, and when to use it.
Not handling tool call errors
Your tool execution can fail. The response to the model should include the error message in the tool result, not an empty response or a generic error. The model can use the error to adjust its next action, but only if you tell it what went wrong.
Forgetting parallel tool calls
GPT-5 can return multiple tool calls in a single response. Your code needs to handle this. A common bug: the code assumes `tool_calls` has exactly one item and breaks when the model calls two tools at once. Check for and handle the array.
The Batch API processes requests asynchronously within 24 hours at half the price of real-time requests. For any workload that doesn't need an immediate response, this is the right API to use.
Use it for:
Batch requests are processed within 24 hours and cost 50% less than real-time requests. The trade-off is latency. Batch is not suitable for any user-facing feature. For background processing jobs that don't need sub-second responses, the cost savings compound significantly over time.
OpenAI costs are easy to ignore until they aren't. Set up monitoring before you need it.
Total API spend is a lagging indicator. What you want to know is which feature is expensive and why. Tag every API call with a feature identifier in the request metadata. Most monitoring tools can aggregate by tag.
A common source of cost spikes: user-supplied content in prompts (e.g., a user pasting a 10,000-word document into a chat). Set token count alerts before costs appear in billing. Truncation or summarization strategies should trigger before you send an unusually large prompt.
OpenAI's rate limits increase with cumulative spend (Tier 1: $100 spent, Tier 2: $500, Tier 3: $1,000, Tier 4: $5,000). If your production launch depends on high rate limits, plan for reaching the required spend tier in advance. You can accelerate by running evals through the API.
Tell us what you're building. We can help you pick the right models, design the integration, and set up cost controls before you launch.