Most AI projects don't fail with a dramatic crash. They fail slowly — with escalating prompt tweaks, user complaints that go unresolved, costs that creep upward, and a growing sense that the system sort of works but not reliably enough to trust.
By the time the failure is obvious, it's expensive to fix. These are the eight signs that appear earlier, before the failure becomes undeniable.
Sign 1: Demo accuracy doesn't match production accuracy
What it looks like: The system performs beautifully in demos, on your curated test cases, and in the staging environment. In production, users report wrong answers, missed context, and failures on inputs that seem straightforward.
What it means: Your evaluation set is too small, too clean, or too similar to your training data. You tested with inputs you chose; production exposes inputs you didn't anticipate. Distribution shift between your test set and real traffic is the most common cause of this gap.
What to do: Build a production sample — take 200–500 real production queries and run them through your eval pipeline. Score them honestly. The gap between your curated accuracy and your production sample accuracy tells you how much work is left. Going forward, randomly sample 1–5% of production queries daily and add them to your eval set as they reveal new failure modes.
Sign 2: Latency gets worse as you add features
What it looks like: You started with a response time of 2–3 seconds. After adding retrieval, conversation history, a second prompt call for formatting, and a third for safety checking, you're at 12–15 seconds. Users are dropping off.
What it means: Features are being added linearly to a pipeline that wasn't designed for compound operations. Each new capability is duct-taped to the existing flow rather than integrated with the architecture in mind.
What to do: Profile each step in the pipeline. Identify where the time is going. Common culprits: retrieval with no caching (embedding the same queries repeatedly), sequential LLM calls that could be parallelized, large context windows passed to expensive models when a cheaper model would work, and synchronous operations that could be async. Once you have the profile, optimize the top two bottlenecks — that usually recovers 60–70% of the latency.
Sign 3: Cost per query rises without volume growth
What it looks like: Your API bill is growing 20–30% month-over-month, but your request volume isn't. The cost per query is climbing.
What it means: Context window creep. Someone added conversation history, added retrieval results, added a longer system prompt, or added a second model call — and no one is watching what these additions cost per query. It adds up faster than people expect.
What to do: Add per-query cost tracking. Tools like Helicone or LangSmith can do this automatically. Once you have visibility, audit the top 10% most expensive queries. Look at what's driving their token counts. Conversation history is usually the biggest offender — implement a summarization strategy for long conversations rather than passing full history. Trim retrieval results to only the chunks that score above a relevance threshold. Review whether the system prompt has grown through accretion of instructions that may not all be necessary.
Sign 4: The team keeps tweaking prompts with no baseline to measure against
What it looks like: Engineers make prompt changes and ship them because "it seemed better in my testing." There are no automated evals. Whether the change actually improved things — or introduced new failure modes while fixing the old one — is unknown.
What it means: You're flying blind. Prompt engineering without an eval harness is just creative writing. Some changes will help some queries and hurt others. Without measurement, you can't tell.
What to do: Stop shipping prompt changes without evals. Build a minimum viable eval set: 50–100 representative queries with expected outputs. Run every prompt change against this set before merging. The eval doesn't need to be perfect — even a simple comparison of "does the new prompt pass at least as many tests as the old one?" is infinitely better than no measurement. Tools like LangSmith, Promptfoo, or even a spreadsheet can serve as the starting point. If you need help setting this up properly, this is exactly the kind of work AI agent evaluation addresses.
Sign 5: Hallucinations are reported by users but you have no eval to reproduce them
What it looks like: A user submits a support ticket: "The AI told me your product supports feature X. It doesn't." You try to reproduce the hallucination. You can't. You tell the user it was a one-off. It wasn't.
What it means: Hallucinations are happening, but you have no systematic way to catch them because you're not logging and evaluating production outputs. You only know about failures that users bother to report — which is a small fraction of actual failures.
What to do: First, implement logging that captures every query and response. Second, add automatic hallucination detection for the most common categories of factual claims in your domain — if your system makes claims about pricing, features, or policies, you can write deterministic checks for those. Third, build a feedback mechanism that makes it easy for users to flag incorrect answers and routes those flags into your eval set. Every user-reported failure should become a permanent test case.
Sign 6: The vendor promises "we'll add RAG later"
What it looks like: You're evaluating an AI vendor's chatbot platform. It doesn't seem to know your product well. Their sales team says: "Right now it's using the general model, but once we integrate your docs, the accuracy will be much better."
What it means: RAG is not a simple upgrade — it's a substantive engineering project. "We'll add it later" usually means it hasn't been built yet, there's no clear timeline, and you'll be on the general-purpose model for longer than you expect. The accuracy problem you're seeing in the demo will be your production experience.
What to do: Ask for a production reference customer who's running the system with their own knowledge base integrated. If they can't provide one, or if all their references are using the general model, the product isn't ready for your use case. Either negotiate a contract that doesn't start billing until RAG is live and validated, or find a vendor whose retrieval pipeline is already production-tested. If the accuracy of knowledge-specific responses is critical to your use case, a purpose-built RAG system may serve you better than waiting for a platform feature.
Sign 7: No one owns the eval dataset
What it looks like: When you ask "who owns the test cases for this AI system?" you get vague answers. There's a spreadsheet somewhere. The ML engineer has some local scripts. The product team added a few cases once. No one considers this their job.
What it means: Evals are the feedback loop that tells you whether the system is getting better or worse over time. Without an owner, the dataset doesn't grow, old cases don't get reviewed for relevance, and regressions from model updates or prompt changes go undetected. The AI project is running on trust that nothing broke.
What to do: Assign a specific person — one person — as the owner of the eval dataset. Define what they're responsible for: adding new cases when failures are found, reviewing the set quarterly, running evals before any significant change ships. If your team is too small for a dedicated ML engineer, this is a role for whoever is most technical and most hands-on with the system's outputs. The eval dataset should live in version control alongside the code.
Sign 8: The AI team can't explain why a specific answer was wrong
What it looks like: A user asks why the AI gave a specific wrong answer. The team's response is: "We're not sure, the model just does that sometimes" or "It's hard to know without being able to reproduce it."
What it means: The system has no observability. There's no trace of which documents were retrieved, which prompt was sent, which model version was used, or what the full context looked like when the failure occurred. Without this, debugging is guesswork and you can't make targeted improvements.
What to do: Implement structured logging that captures, for every query: the full prompt (including system prompt), the retrieved documents and their scores (if using RAG), the model and version, the full response, and a unique trace ID that links all of this together. Tools like LangSmith, Langfuse, or Weights & Biases can handle this. Once you have traces, any reported failure should be debuggable in under five minutes by pulling the trace. If you can't explain a failure from the logs, that's a gap in your logging, not an inherent mystery.
If you're reading this list and recognizing three or more signs in a current project, that's not unusual — many AI projects ship without the operational infrastructure in place, and add it later under pressure. The earlier you address these, the cheaper the fix.
If you're inheriting a project with multiple signs, or if you're trying to decide whether a struggling project is salvageable, a structured evaluation of the existing system is usually the right first step before deciding whether to repair or replace. Similarly, if a project has already failed and needs to be rebuilt with the right foundation, the path forward often looks different from what went wrong originally — see how we approach failed AI project rescues.
The bottom line
The most recoverable AI projects are the ones where problems are caught at signs 1–4. Signs 5–8 suggest the operational infrastructure was never built, which means the system has been running without a safety net.
Evals, logging, ownership, and cost visibility aren't nice-to-haves. They're the difference between an AI system you can manage and one you're hoping doesn't break.