Retrieval-Augmented Generation is now a well-understood pattern, but a lot of developers who list it on their resume have only built a tutorial demo — not a production system handling real queries from real users. The gap between "built a RAG prototype" and "shipped a RAG system that actually works" is significant, and this guide will help you find the second kind.
What a competent RAG developer needs to know
A developer who can build a production RAG system has working knowledge across four areas. Use this as your evaluation framework.
Vector databases and embedding models
They should know the major vector database options (Pinecone, Weaviate, Qdrant, pgvector) and when to use each. Pinecone is fully managed and fast to start with but expensive at scale. pgvector is fine for smaller corpora and reduces infrastructure complexity. Weaviate and Qdrant give you self-hosted control with good performance.
They should understand the trade-offs between embedding models: OpenAI text-embedding-3-small is cheap and widely used; text-embedding-3-large is higher quality but costs 5x more; Cohere's embed-v3 outperforms OpenAI on some retrieval benchmarks; local models like bge-m3 or nomic-embed-text eliminate per-token costs but require infrastructure.
A developer who can't name at least three embedding models and explain when they'd use each hasn't thought carefully about retrieval quality.
Chunking strategies
This is where many RAG implementations break. Getting chunking wrong is one of the most common reasons a RAG system returns irrelevant or truncated context.
A capable developer will know:
- Fixed-size chunking (e.g., 512 tokens with 50-token overlap) — simple but loses context at chunk boundaries
- Semantic chunking — splits on natural semantic breaks rather than token counts; better for prose
- Document-structure-aware chunking — preserves headings, tables, and sections as natural boundaries
- Small-to-big retrieval — stores small chunks for retrieval but passes a larger parent chunk to the LLM for context
Ask them: "How would you chunk a 200-page technical manual with tables, headers, and footnotes?" A generic answer ("I'd split it into chunks") is a red flag. A specific answer that mentions structure preservation, table handling, and metadata attachment is a good sign.
Reranking
Many RAG implementations stop at initial vector retrieval. Production systems usually need a reranking step that re-scores retrieved candidates using a more expensive cross-encoder model before passing results to the LLM.
Good rerankers: Cohere Rerank, BGE-reranker, Jina Reranker. Each adds latency (typically 100–300ms) but meaningfully improves relevance, especially for complex queries.
A developer who has shipped production RAG will know that initial cosine similarity retrieval is often not good enough and will have opinions on when reranking is worth the latency cost.
Eval frameworks
This is the most reliable separator between developers who have shipped to production and those who have not.
A production RAG developer will know how to measure:
- Recall@k: of the relevant documents in the corpus, how many are in the top-k retrieved results?
- Context precision: of what's retrieved, how much is actually relevant?
- Faithfulness: does the generated answer stick to the retrieved context, or does it hallucinate?
- Answer relevance: does the answer actually respond to the question?
The standard tooling for this is Ragas, though some teams build custom harnesses. BEIR benchmarks are used for model-level comparisons. TruLens is another option.
A developer who can't describe how they'd measure retrieval quality has never tried to improve it systematically.
Interview questions that reveal real experience
These questions don't have one right answer, but they separate practitioners from tutorial-followers.
"How do you measure retrieval accuracy, and what do you do if recall@k falls below 0.7 on your eval set?"
A developer who has shipped production RAG will have an answer: check chunk quality, check query-document mismatch (query is short, document is long), try hybrid search (vector + BM25), add a reranking step, re-embed with a better model. A developer who hasn't will give a vague answer about "tuning."
"Walk me through a time your RAG system returned wrong answers in production and how you diagnosed it."
Production systems break. The question isn't whether they've seen failures — it's whether they've investigated them systematically. Look for: checking retrieved chunks to see if retrieval was the problem vs. the LLM's generation, checking whether the ground-truth answer was even in the corpus, using span-level logging to trace where the answer came from.
"What's your default chunking strategy and when would you deviate from it?"
A specific answer with a named default (e.g., "I start with 512 tokens/50 overlap, but for structured documents I split on section headers") plus an explanation of when it breaks (long tables, code blocks, multilingual text) indicates genuine experience.
"How do you handle the cold-start problem when you don't have historical queries to build an eval set from?"
Good answer: use the LLM itself to generate synthetic question/answer pairs from the corpus, then manually validate a sample. Or use a golden set from subject-matter experts. Bad answer: "We test it with some queries to make sure it works."
"What's your approach to preventing the LLM from hallucinating beyond the retrieved context?"
Look for: explicit prompting to cite sources, retrieval-only answers (force the model to answer only from context), citation grounding verification, confidence thresholds that trigger a "I don't know" response when retrieval confidence is low.
Portfolio signals
When reviewing work history, look for:
Has the system handled real query volume in production? A system that "processed thousands of documents" is table-stakes. A system that "handles 10,000 queries per day with a p95 latency of 800ms" tells you it's been load-tested.
Is there any mention of eval metrics? If a developer can't tell you what the retrieval accuracy was on their last RAG project, they weren't measuring it.
Did they deal with multiple data types? PDFs, HTML, tables, code, images. A system that only handles clean paragraph text is simpler than one that handles mixed-format enterprise documents.
Do they mention what broke? Developers who have been in production talk about what went wrong. Chunking strategy had to be redesigned because tables were getting split. Embedding model was replaced because it performed poorly on domain-specific terminology. Reranking was added because initial retrieval had too many false positives. These specifics indicate real operational experience.
What to expect to pay
As of Q2 2025, realistic market rates:
Freelance RAG developer (contract): $120–$200/hr. Below $120/hr usually means junior or offshore. Above $200/hr is typically someone with a specialized track record (healthcare data, legal, finance).
Agency with RAG specialty: $15,000–$25,000/month for an ongoing engagement. Fixed-scope projects range from $25,000 to $80,000+ depending on data complexity, the number of data sources, and the eval rigor required.
Staff augmentation (embedded contractor): $180–$250/day on platforms like Toptal or direct sourcing. Budget for a minimum 3-month commitment to get meaningful output.
The cheap end of these ranges is not wrong on its face — there are skilled offshore developers who build good RAG systems. But if you're paying the cheap rate, you need to do more of your own quality oversight. That means running evals yourself, reviewing chunking decisions, and pressure-testing the system before it goes to users.
Engagement structure that works
A RAG project typically has three phases, and the contract should reflect them:
Phase 1 — Data and eval setup (2–4 weeks): Ingest a representative sample of your documents, establish a golden eval set of 50–100 question/answer pairs, measure baseline retrieval quality. This phase should end with a written eval report. If a vendor skips this phase, every optimization decision in Phase 2 will be arbitrary.
Phase 2 — System build and optimization (4–10 weeks): Build the full pipeline, optimize chunking and retrieval for your data distribution, hit agreed-upon eval targets. Gate completion on the eval numbers, not calendar time.
Phase 3 — Deployment and handoff (1–2 weeks): Production deployment, monitoring setup, documentation, knowledge transfer. The handoff should leave you able to run evals independently when the system needs to be updated.
Push back on any proposal that doesn't include an explicit eval gate between Phase 1 and Phase 2. Without baseline metrics, you can't know if the system is getting better or worse.
The bottom line
Hiring a RAG developer comes down to one question: can they tell you what the retrieval accuracy is, and do they know how to improve it when it's not good enough?
The interview questions above will reveal whether they have systematic eval discipline or are operating by intuition. Portfolio signals will tell you whether they've dealt with production realities. The engagement structure should enforce measurement gates so you always know whether you're getting what you paid for.
If you're evaluating RAG for your organization and want to understand what a well-measured system looks like, our RAG development services page describes the approach we take, including the eval frameworks we use on every project.