Loading…
Rescue
Hallucination in RAG systems comes from three sources: retrieval returning the wrong chunks, the LLM confabulating when the right chunk isn't available, or the prompt not constraining the model tightly enough. Most teams treating all wrong answers as “hallucination” are solving the wrong problem.
We run an eval against your system, identify which failure mode is dominant, and fix it. Most RAG systems we audit have retrieval problems, not generation problems, the model is doing its job. The wrong context is just being handed to it.
Tell us what broke.
Every wrong answer in a RAG system traces back to one of these three. Most systems have one dominant cause. Knowing which one determines what gets fixed.
The right chunk is somewhere in your corpus but it is not being returned. This is the most common root cause, and the one most teams misdiagnose as a 'hallucination problem'. Common causes: chunk size too large (the signal gets diluted across a 2,000-token chunk when it lives in 200 words), embedding model mismatch between indexing and query time, no reranking step to promote the most relevant results above noise, or a similarity threshold set so low that the top-k results are only loosely related to the query. Fix: tuning the chunking strategy to match the density of information in your documents, switching to a better-calibrated embedding model, adding a cross-encoder reranker as a second pass, and raising the similarity threshold with an explicit fallback path for queries where nothing scores highly enough.
Multiple chunks are retrieved, but the way they are assembled and passed to the LLM creates confusion rather than clarity. Common causes: no deduplication step (the same paragraph appears in three documents and gets retrieved three times), conflicting information from documents with different dates (a policy updated in March contradicts the version from January, both are in the context window), metadata not included alongside the text (the LLM has no way to know which document a chunk came from or when it was written). Fix: deduplication before assembly, date-weighted retrieval that prioritizes recent documents when recency is relevant, and including document source and date metadata in the context so the LLM can reason about conflicts.
The right context is present in the prompt, but the LLM ignores it or supplements it with information from its training data. This is the least common root cause but the one that gets blamed most often. Common causes: system prompt too permissive (no explicit instruction to only use provided context), model temperature too high for a factual task, prompt doesn't instruct the model to cite its source or say 'I don't know' when the answer isn't in the context. Fix: a tighter system prompt with explicit instructions to use only the provided context and cite the source document, temperature reduced to 0 or 0.1 for factual retrieval tasks, and an optional verification step that checks each claim in the generated answer against the retrieved chunks.
Every engagement starts and ends with measurement. Fixes without measurement are guesswork. We don't do guesswork.
We build a golden-set test suite from your real production queries (the questions your users are actually asking) and score the current system on retrieval accuracy and answer faithfulness. This is the number we measure against.
Written report identifying which failure mode dominates your error cases and why. We don't give you a list of everything that might be wrong. We tell you what is wrong and which fix will have the most impact.
We implement the fixes that address the diagnosed root cause: chunking strategy, reranking, prompt changes, or a combination. We don't fix everything speculatively. We fix what the eval data shows is actually broken.
The same golden-set test suite, run again after fixes, scored against the same criteria. You get a before/after number. Not an estimate. An actual measurement on your actual data.
A document you can share with your team, your board, or your customers showing exactly what the system accuracy was before we started and what it is now.
Model providers change model behavior without notice. An ongoing monthly eval run catches drift before your users do. We run the golden set monthly and alert you when scores drop.
Most RAG systems with retrieval problems go from 60–70% accuracy to 85–92% after targeted fixes. That is a meaningful improvement and it is achievable in most cases we take on.
Getting above 95% requires either a very narrow query scope, a high-quality curated document corpus, or both. If your chatbot covers ten tightly defined topics and all your documents were written by the same team with consistent formatting, 95% is realistic. If it covers 50 topics across 1,000 documents with inconsistent structure and outdated entries, 92% is closer to the realistic ceiling.
We won't promise 100%. Any vendor promising 100% accuracy on a RAG system is not measuring it properly.
Some projects don't have what this engagement needs to produce a real result.
Systems with fewer than 500 real user queries
We need real queries from real users to build a meaningful golden set. Synthetic queries built to look like user queries produce an eval that passes but doesn't reflect actual performance. If your system is too new to have that data, come back when it does.
Teams who want to skip measurement and jump to fixes
Every engagement starts with a baseline eval run. If you want us to change the chunking strategy without measuring whether the current strategy is actually the problem, we won't do it. Fixes without measurement are guesswork, and guesswork makes things worse as often as it makes them better.
Chatbots on third-party platforms you can't modify at the retrieval level
If your chatbot runs on Chatbase, Intercom Fin, or Zendesk Answer Bot and you don't have access to the retrieval layer, there is nothing to fix at the engineering level. The fixes that matter (chunking, reranking, prompt) require access to the underlying system.