RAG & GraphRAG System Design

RAG & GraphRAG System Design Interview

Preview: https://drive.google.com/file/d/1IzCcRv6rkqQVPgD_xHzwVx3yXhk7Nlv3/view?usp=sharing

Chapter 1: RAG Foundations and System Design Fundamentals

Q1: [System Design] Design a document question answering system for 50,000 internal documents

Category: System Design

Question: Walk me through your architecture, and justify the components you include as well as the ones you leave out.

Q2: [Production Debugging] Answers are confident and wrong. How do you localize the fault?

Category: Production Debugging

Question: A user reports that the assistant invents policy details. You have logs, the index, and the ability to replay requests. Where do you look, and in what order?

Q3: [Technical Depth] Why not simply use a one million token context window?

Category: Technical Depth

Question: Your leadership asks why the team is building retrieval infrastructure when the newest model accepts a million tokens.

Q4: [Case Study] The customer support bot regressed after a model upgrade

Category: Case Study

Question: Retrieval quality dropped 30 points immediately after the team upgraded the embedding model. Nothing else changed. Diagnose it.

Q5: [Trade-offs] How do you choose K, and what breaks at the extremes?

Category: Trade-offs

Question: Your retriever returns the top K chunks. Explain how you would select K empirically and what fails when K is too small or too large.

Q6: [Production] Design the abstention policy and defend it against product pressure

Category: Production

Question: Product management complains that the assistant refuses too often. Engineering complains that lowering the threshold causes hallucination. Resolve this.

Q7: [Scalability] Take this design from 50,000 documents to 10 million

Category: Scalability

Question: Same product, two hundred times the corpus. What changes, and what stays the same?

Chapter 2: Document Ingestion, Parsing, and Chunking

Q8: [System Design] Design the ingestion pipeline for a 5 million document corpus with mixed formats

Category: System Design

Question: Documents arrive continuously. Formats include scanned PDFs, native PDFs, DOCX, Confluence HTML, and CSV exports. Design the pipeline end to end.

Q9: [Technical Depth] Explain how you would select chunk size empirically

Category: Technical Depth

Question: Do not give me a number. Give me the method that produces the number.

Q10: [Production Debugging] Retrieval works for most documents and fails completely for a subset

Category: Production Debugging

Question: Recall is 0.86 overall. For one document class it is 0.09. Where do you look?

Q11: [Case Study] A wiki assistant returns the same navigation text for every question

Category: Case Study

Question: Every answer cites the same three chunks, which contain the site menu. Explain the mechanism and the fix.

Q12: [Trade-offs] When is semantic chunking worth 40 times the ingestion cost?

Category: Trade-offs

Question: Defend or reject semantic chunking for a specific workload.

Q13: [Production] How do you handle document updates and deletions without reindexing everything?

Category: Production

Question: A 5 million document corpus changes by 2 percent daily. Full reindexing takes 40 hours. Design incremental maintenance.

Q14: [Technical Depth] Why does adding the section heading to each chunk improve retrieval so much?

Category: Technical Depth

Question: Explain the mechanism, not just the empirical result.

Chapter 3: Embeddings and Vector Database Architecture

Q15: [System Design] Design the storage layer for a multi-tenant RAG platform with 2,000 tenants

Category: System Design

Question: Tenants range from 100 documents to 400,000 documents. Isolation is a contractual requirement. Design the vector storage.

Q16: [Technical Depth] Explain HNSW well enough that I could implement search over it

Category: Technical Depth

Question: Describe the data structure and the search procedure precisely.

Q17: [Production Debugging] Search returns nothing for a specific customer, yet their documents are indexed

Category: Production Debugging

Question: Other customers work. The index contains their rows. Diagnose it.

Q18: [Case Study] Recall dropped after migrating from exhaustive search to HNSW

Category: Case Study

Question: Accuracy fell four points after the index change. Leadership wants the index reverted. What do you do?

Q19: [Trade-offs] Would you choose pgvector or a dedicated vector database?

Category: Trade-offs

Question: Justify the choice for a team of eight engineers serving 3 million chunks.

Q20: [Production] How do you upgrade the embedding model with zero downtime?

Category: Production

Question: Your team wants to move from a 768 dimensional model to a better 1024 dimensional one over a 25 million chunk index.

Q21: [Technical Depth] Why do similarity scores from different models mean different things?

Category: Technical Depth

Question: A colleague proposes alerting when similarity falls below 0.7. Evaluate that proposal.

Chapter 4: Building a Complete Basic RAG System

Q22: [System Design] Design a conversational RAG service with a strict 2 second p95

Category: System Design

Question: Include the latency budget, the caching strategy, and what you drop under load.

Q23: [Production Debugging] Citations point at the wrong sources

Category: Production Debugging

Question: The answers are factually correct, yet the bracketed numbers reference blocks that do not contain the cited claim. Diagnose and fix it.

Q24: [Technical Depth] Why does placing the best chunk in the middle of the prompt hurt accuracy?

Category: Technical Depth

Question: Explain the mechanism and how your context builder responds to it.

Q25: [Case Study] Answers degrade only for long conversations

Category: Case Study

Question: Turn one is excellent. Turn eight is poor. Cost per turn has also tripled. Explain.

Q26: [Trade-offs] Should retrieval failure produce a refusal or a best effort answer?

Category: Trade-offs

Question: Product wants an answer every time. Compliance wants a refusal whenever evidence is weak. Decide.

Q27: [Production] How do you version prompts and roll them out safely?

Category: Production

Question: A prompt change improved your test set and regressed production. Design the process that prevents recurrence.

Q28: [Technical Depth] Walk me through everything that happens between the user pressing enter and the first token appearing

Category: Technical Depth

Question: Be specific about what is parallelizable and where the time actually goes.

Chapter 5: Advanced Retrieval and Hybrid RAG

Q29: [System Design] Design a hybrid retrieval service that stays within a 400 millisecond retrieval budget

Category: System Design

Question: Include dense, lexical, fusion, and reranking. Show where the time goes and what you cut first.

Q30: [Technical Depth] Why fuse ranks instead of normalizing and adding scores?

Category: Technical Depth

Question: Defend reciprocal rank fusion against weighted score combination.

Q31: [Production Debugging] Hybrid retrieval performs worse than dense alone

Category: Production Debugging

Question: Adding BM25 dropped your golden set recall by three points. Explain how that is possible and how you fix it.

Q32: [Case Study] A legal research tool misses relevant precedents that a paralegal finds in seconds

Category: Case Study

Question: The corpus is complete. The retriever is dense with reranking. Diagnose it.

Q33: [Trade-offs] When would you skip reranking entirely?

Category: Trade-offs

Question: Reranking is the standard recommendation. Argue the other side.

Q34: [Production] Design the query classifier that drives adaptive routing

Category: Production

Question: It must be fast, cheap, and correct enough to route. How do you build and monitor it?

Q35: [Technical Depth] Explain corrective RAG and where it belongs relative to answer verification

Category: Technical Depth

Question: Both check quality. Why have both?

Chapter 6: RAG Evaluation, Observability, and Debugging

Q36: [System Design] Design the evaluation system for a RAG platform serving 40 teams

Category: System Design

Question: Each team has its own corpus and its own quality bar. Build the evaluation infrastructure.

Q37: [Technical Depth] How do you validate that your LLM judge is trustworthy?

Category: Technical Depth

Question: Your faithfulness metric depends entirely on the judge. Prove it works.

Q38: [Production Debugging] Offline metrics improved and users complained

Category: Production Debugging

Question: Faithfulness rose two points and the thumbs-down rate doubled. Explain the disconnect.

Q39: [Case Study] Retrieval metrics look excellent and answers are still wrong

Category: Case Study

Question: Recall@5 is 0.94, MRR is 0.88, faithfulness is 0.91, and users report incorrect answers. Investigate.

Q40: [Trade-offs] How much evaluation is enough before shipping?

Category: Trade-offs

Question: Full evaluation takes 90 minutes. Engineers want to merge in 10. Resolve it.

Q41: [Production] Design the observability stack for a RAG system

Category: Production

Question: Assume no ground truth labels in production. What do you log, and what do you alert on?

Q42: [Technical Depth] Explain why claim level faithfulness beats paragraph level scoring

Category: Technical Depth

Question: Both use the same judge model. Why does decomposition help?

Chapter 7: Knowledge Graph Fundamentals

Q43: [System Design] Design the knowledge graph layer for an enterprise with 200,000 documents

Category: System Design

Question: Documents span contracts, incident reports, and org charts. Design extraction, storage, and maintenance.

Q44: [Technical Depth] Why is a graph database faster than SQL for multi-hop queries?

Category: Technical Depth

Question: Both can model a graph. Explain the performance difference precisely.

Q45: [Production Debugging] The extracted graph is fragmented and traversals return nothing

Category: Production Debugging

Question: The graph has 80,000 nodes and 12,000 relationships. Diagnose it.

Q46: [Case Study] An LLM built a graph with 340 relationship types

Category: Case Study

Question: Queries are unwritable. Explain how this happened and how you recover without re-extracting everything.

Q47: [Trade-offs] When is a knowledge graph not worth building?

Category: Trade-offs

Question: Argue against graph adoption for a specific system.

Q48: [Production] How do you keep the graph synchronized with changing documents?

Category: Production

Question: Documents update daily. The graph must reflect current reality without full rebuilds.

Q49: [Technical Depth] How do you prevent Cypher injection when relationship types come from an LLM?

Category: Technical Depth

Question: Your extractor emits a type string that becomes part of a query. Secure it.

Chapter 8: GraphRAG and Hybrid Graph Retrieval

Q50: [System Design] Design a GraphRAG system for supply chain risk analysis

Category: System Design

Question: Users ask which customers are exposed when a supplier fails. Design it end to end.

Q51: [Technical Depth] Explain local search and global search, and when each fails

Category: Technical Depth

Question: Be specific about the mechanism, not just the intuition.

Q52: [Production Debugging] GraphRAG returns empty context for half of all queries

Category: Production Debugging

Question: The graph is well populated. Traversals work when you test them manually. Diagnose it.

Q53: [Case Study] A three hop query brought down the production Neo4j instance

Category: Case Study

Question: The query had a LIMIT clause. Explain why the limit did not protect you.

Q54: [Trade-offs] Hybrid GraphRAG or advanced vector RAG for a customer support assistant?

Category: Trade-offs

Question: Pick one and defend it.

Q55: [Production] How do you keep community summaries fresh without full recomputation?

Category: Production

Question: The graph changes hourly. Full community detection and summarization takes six hours.

Q56: [Technical Depth] Why does concatenating graph and vector context often make answers worse?

Category: Technical Depth

Question: Both contexts are relevant. Explain the degradation.

Chapter 9: Agentic GraphRAG and Multi-Hop Reasoning

Q57: [System Design] Design an agentic research assistant that answers multi-hop questions

Category: System Design

Question: It must handle questions requiring two to four retrieval steps, stay under 15 seconds, and never loop forever.

Q58: [Technical Depth] Explain LangGraph state reducers and why they matter

Category: Technical Depth

Question: Be specific about the concurrency semantics.

Q59: [Production Debugging] The agent loops until the cap on most queries

Category: Production Debugging

Question: Latency is terrible and answers are no better than a fixed pipeline. Diagnose it.

Q60: [Case Study] An agent gave a different answer to the same question twice in a row

Category: Case Study

Question: Same corpus, same question, same user. Explain the non-determinism and how to control it.

Q61: [Trade-offs] Fixed pipeline, router, or full agent?

Category: Trade-offs

Question: Choose an architecture for an enterprise knowledge assistant and defend it.

Q62: [Production] How do you test an agentic workflow?

Category: Production

Question: Outputs are non-deterministic. Traditional assertions do not apply.

Q63: [Technical Depth] Where does human-in-the-loop belong, and how do you implement it without blocking?

Category: Technical Depth

Question: Design the escalation path for a regulated use case.

Chapter 10: Production RAG and GraphRAG System Design Interviews

Q64: [System Design] Design a RAG platform serving 50 million documents and 2,000 queries per second

Category: System Design

Question: Give me the full architecture, the numbers, and the parts you would build last.

Q65: [Production] Your p95 latency doubled overnight with no deployment

Category: Production

Question: Nothing shipped. Traffic is flat. Diagnose it.

Q66: [Case Study] The board asks why the RAG project costs three times its forecast

Category: Case Study

Question: Present the analysis and the plan.

Q67: [Technical Depth] How do you enforce document-level access control in retrieval?

Category: Technical Depth

Question: Users have different permissions. The same query must return different evidence per user.

Q68: [Trade-offs] Local models or hosted API models for an enterprise deployment?

Category: Trade-offs

Question: Justify the choice with more than a preference.

Q69: [Production] Design the rollout plan for replacing a keyword search system with RAG

Category: Production

Question: Ten thousand internal users depend on the current system daily.

Q70: [System Design] Walk me through everything you would monitor, and what each alert means

Category: System Design

Question: Assume the system is live and you are on call.