AI Agent Development · San Francisco

AI agent development in San Francisco.

The hard part of an agent project in the Bay is not picking a framework. The hard part is designing the agent loop so that tool calls stay reliable past a 10-step trajectory, designing the tool schemas so function calling does not drift, designing the memory layer so the agent recovers from a restart, and designing the eval harness so a prompt change does not silently regress a use case you cared about three weeks ago.

We build agents for SaaS, AI infrastructure, and dev tooling companies in the city. LangGraph for the state graph, Claude Sonnet 5 or GPT-5 for the reasoning step, Anthropic tool use or OpenAI Responses API for the action layer, Langfuse for traces, Braintrust for offline evals, pgvector or Pinecone for memory. We pick by benchmark on your workload, not by vendor preference.

Projects run fixed scope. Discovery first, fixed quote after, four to eight weeks of build, weekly demos on Pacific time.

How we work

Tell us about the agent you're building.

Agent loop architecture, in detail.

Most production agents we ship follow a plan-and-execute graph with explicit state. A planner step produces a typed plan (an array of action objects), an executor step runs each action through a tool-use call, and a verifier step checks the result against a schema before transitioning to the next node. We use LangGraph because the state machine is the source of truth, not a hidden prompt convention. A senior engineer on your team can read the graph and know exactly what the agent will do.

We choose ReAct when the tool space is small (under five tools) and the workflow benefits from interleaved reasoning and action. ReAct fits well for code-search agents, internal Q&A bots that hit two or three APIs, and SRE diagnostic agents. We avoid ReAct at higher tool counts because tool-selection error compounds across steps.

Reflection patterns (Reflexion, self-critique) only earn their cost when there is a ground-truth verifier. A test suite, a production database query, an API that returns a status code. Without a verifier, self-critique is the model talking to itself and getting more confident in the wrong answer.

Tool-use design is where most agent projects go sideways. The Anthropic tool-use API and the OpenAI Responses API both expose JSON Schema for arguments, and both models handle flat schemas with primitive types far more reliably than nested unions or discriminated types. Every tool we ship has a flat argument schema, a short positive example in the description, and a forbidden-example callout that names the most common hallucination mode.

Tool outputs are validated with Zod or Pydantic before they go back into the agent context. A 500 error from your internal API becomes a typed error object, not a string the model has to parse. This single discipline removes most of the silent failure modes we see when teams hand-roll tool integrations.

For multi-agent coordination, the orchestrator-worker pattern from the Anthropic research-agent blog post is the most reliable shape we have seen. We use it when the workflow has parallelism (gather from N sources, synthesize one answer) and avoid it when a single agent with the right tools is enough.

What we build for San Francisco teams.

Six engineering components that appear in every production agent we ship for SaaS, AI labs, and dev tooling companies.

State graph with explicit transitions

LangGraph for stateful workflows. Each node has a typed input and output, transitions are deterministic where possible, and the graph is checkpointed so a restart resumes from the last completed step instead of replaying tokens.

Tool schema design

Flat argument schemas, positive and forbidden examples in descriptions, Zod or Pydantic validation on every tool output. Tool count per call kept under eight to preserve selection accuracy on Claude Sonnet 5 and GPT-5.

Memory pattern that fits the workflow

Short-term context in the graph state, long-term in pgvector or Pinecone keyed by tenant and session. We pick session memory, semantic memory, or hybrid based on the access pattern, not because every agent needs all three.

Multi-agent orchestration when warranted

Orchestrator-worker pattern using LangGraph subgraphs or AutoGen group chat. We deploy this only when parallel gathering or genuine specialization beats a single agent with more tools.

Observability with Langfuse or Helicone

Every prompt, tool call, latency number, and token cost is captured. Traces are session-grouped and searchable by user, tenant, and trajectory length. Cost dashboards by feature so the finance team can model unit economics.

Eval harness gating CI

Braintrust or Promptfoo against a labeled set we build with you. Task completion rate, hallucination rate, tool-call accuracy, latency p50 and p95. Regression suite runs on every prompt or model change before merge.

Why this work fits San Francisco.

San Francisco is the center of the agent industry. Anthropic and OpenAI publish from the city, Salesforce ships Agentforce out of Mission Bay, Stripe runs agents in their dispute-review and support flows, Figma and Notion have shipped agentic features in the last year, and most of the late-stage SaaS companies in the city now have an agents team. The bar is high because every team has read the same papers and tried the same patterns.

The teams we work with in the city tend to have a strong product and ML background but a thin layer of production-agent operating experience. They know the model APIs, they have built prototypes, and they need help shipping something that holds up against an eval set, an observability dashboard, and a real user load. That is where we are useful: the boring parts (graph design, tool validation, eval pipeline, cost monitoring) that decide whether the prototype survives contact with production.

We work remotely with Bay Area clients on Pacific time. Discovery calls fit between 9am and 5pm PT, demos happen weekly, and the codebase ships to a repository you own.

Industries where we see strongest fit: B2B SaaS adding agentic workflows, AI infrastructure companies shipping reference implementations, dev tooling building agentic IDE or CLI features, and Series B+ startups with one agent in production that needs to scale to ten.

Technical questions from San Francisco teams.

Which agent loop pattern do you default to, and when do you deviate?: We default to plan-and-execute with a structured planner step and a typed action schema, because it produces the cleanest audit trail and the most reliable evals. We move to ReAct when the workflow is exploratory and the tool space is small enough that interleaved reasoning beats upfront planning. We use reflection (Reflexion-style) only when there is a clear verifier signal (a unit test passes, an API returns 200, a query returns a row) that the agent can ground self-critique on. We avoid open-ended loops without termination conditions because they burn tokens and produce worse evals than a constrained graph.
How do you design tool schemas so function calling stays reliable?: Three rules. First, every tool has a flat argument schema with no nested unions, because models reliably hallucinate one-of branches. Second, every argument has a short description that includes a positive example and a forbidden example, because the model uses those as in-context calibration. Third, tool outputs are typed and validated with Zod (TypeScript) or Pydantic (Python) before they go back into the context, so a malformed response from your API never enters the model's reasoning. We also keep the tool count per call below eight when possible because selection accuracy degrades above that on most models.
What do you use for agent observability and evals?: Langfuse for traces and session inspection, Braintrust for offline evals and CI gating, and Promptfoo for quick prompt-variant comparisons. Langfuse lives in the runtime path so every prompt, tool call, and token-cost is captured. Braintrust holds the labeled eval set and runs the regression suite on every prompt or model change before merge. Helicone is a fine alternative for the trace layer if your stack is already on it. We pin the model version (Claude Sonnet 5, gpt-5-2025-08-07, Gemini 2.0 Flash) in every trace because eval drift across model snapshots is the most common silent regression.
When do you use multi-agent (orchestrator and workers) versus a single agent with more tools?: Multi-agent earns its complexity when there is genuine specialization (different system prompts, different tool sets, different context windows) and when the orchestrator can reason about which worker to dispatch. The Anthropic research-agent pattern is the canonical case: an orchestrator that plans, parallel workers that gather, a writer that synthesizes. Outside that pattern, multi-agent usually adds latency, cost, and failure modes without benefit. If a single Claude or GPT-5 call with eight tools can do the work, a single agent is the right shape.

Ship the agent your prototype is trying to be.

Describe what your agent has to do and what is currently breaking about it. We'll reply within one business day with a rough scope, a recommended architecture, and a price range.

AI agent development in San Francisco.

Projects run fixed scope. Discovery first, fixed quote after, four to eight weeks of build, weekly demos on Pacific time.

Agent loop architecture, in detail.

What we build for San Francisco teams.

Six engineering components that appear in every production agent we ship for SaaS, AI labs, and dev tooling companies.

State graph with explicit transitions

Tool schema design

Memory pattern that fits the workflow

Multi-agent orchestration when warranted

Orchestrator-worker pattern using LangGraph subgraphs or AutoGen group chat. We deploy this only when parallel gathering or genuine specialization beats a single agent with more tools.

Observability with Langfuse or Helicone

Eval harness gating CI

Why this work fits San Francisco.

We work remotely with Bay Area clients on Pacific time. Discovery calls fit between 9am and 5pm PT, demos happen weekly, and the codebase ships to a repository you own.

Technical questions from San Francisco teams.

Which agent loop pattern do you default to, and when do you deviate?

We default to plan-and-execute with a structured planner step and a typed action schema, because it produces the cleanest audit trail and the most reliable evals. We move to ReAct when the workflow is exploratory and the tool space is small enough that interleaved reasoning beats upfront planning. We use reflection (Reflexion-style) only when there is a clear verifier signal (a unit test passes, an API returns 200, a query returns a row) that the agent can ground self-critique on. We avoid open-ended loops without termination conditions because they burn tokens and produce worse evals than a constrained graph.

How do you design tool schemas so function calling stays reliable?

Three rules. First, every tool has a flat argument schema with no nested unions, because models reliably hallucinate one-of branches. Second, every argument has a short description that includes a positive example and a forbidden example, because the model uses those as in-context calibration. Third, tool outputs are typed and validated with Zod (TypeScript) or Pydantic (Python) before they go back into the context, so a malformed response from your API never enters the model's reasoning. We also keep the tool count per call below eight when possible because selection accuracy degrades above that on most models.

What do you use for agent observability and evals?

Langfuse for traces and session inspection, Braintrust for offline evals and CI gating, and Promptfoo for quick prompt-variant comparisons. Langfuse lives in the runtime path so every prompt, tool call, and token-cost is captured. Braintrust holds the labeled eval set and runs the regression suite on every prompt or model change before merge. Helicone is a fine alternative for the trace layer if your stack is already on it. We pin the model version (Claude Sonnet 5, gpt-5-2025-08-07, Gemini 2.0 Flash) in every trace because eval drift across model snapshots is the most common silent regression.

When do you use multi-agent (orchestrator and workers) versus a single agent with more tools?

Multi-agent earns its complexity when there is genuine specialization (different system prompts, different tool sets, different context windows) and when the orchestrator can reason about which worker to dispatch. The Anthropic research-agent pattern is the canonical case: an orchestrator that plans, parallel workers that gather, a writer that synthesizes. Outside that pattern, multi-agent usually adds latency, cost, and failure modes without benefit. If a single Claude or GPT-5 call with eight tools can do the work, a single agent is the right shape.

Ship the agent your prototype is trying to be.

Describe what your agent has to do and what is currently breaking about it. We'll reply within one business day with a rough scope, a recommended architecture, and a price range.