{
  "id": "what-is-rag",
  "module": "rag",
  "level": 2,
  "minutes": 25,
  "audiences": [
    "beginner",
    "developer",
    "researcher"
  ],
  "software": [
    "anythingllm",
    "cherry-studio",
    "obsidian"
  ],
  "text": {
    "en": {
      "title": "RAG Explained: Retrieval-Augmented Generation from Scratch",
      "description": "What retrieval-augmented generation is, why it beats fine-tuning for fresh or private knowledge, and how a RAG pipeline works step by step: chunking, embeddings, vector search, hybrid search, re-ranking, prompting with citations and evaluation.",
      "updated": "2026-10-22",
      "takeaways": [
        "RAG retrieves relevant passages from your own documents at question time and puts them in the prompt, so the model answers from evidence instead of memory.",
        "The pipeline has an offline indexing half (load → chunk → embed → store) and an online query half (retrieve → re-rank → generate with citations).",
        "Retrieval quality is the bottleneck: chunking, hybrid keyword + vector search and re-ranking usually matter more than the choice of LLM.",
        "Evaluate retrieval and generation separately: did we fetch the right passages, and is the answer faithful to them?"
      ],
      "faq": [
        {
          "q": "RAG or fine-tuning — which should I use?",
          "a": "Use RAG when the model needs facts that change, are private, or must be cited. Use fine-tuning to change behaviour, format or style. They combine well: a fine-tuned model can still use RAG."
        },
        {
          "q": "Do I need a vector database for RAG?",
          "a": "Not always. Small collections fit in memory or in SQLite/Postgres with a vector extension; dedicated vector databases help when you have millions of chunks, need filtering at scale or many concurrent users. Keyword search (BM25) alone is also a valid retriever."
        },
        {
          "q": "Doesn't a long context window make RAG unnecessary?",
          "a": "Long contexts help, but stuffing everything in is slow and expensive, and models use information in the middle of long inputs less reliably. Retrieval keeps prompts small, cheap and focused, and gives you citations."
        },
        {
          "q": "Why does my RAG system still hallucinate?",
          "a": "Usually because retrieval missed the right passage, the chunk lacked context, or the prompt did not instruct the model to answer only from the sources and say when they are insufficient."
        }
      ],
      "sources": [
        {
          "title": "Lewis et al. (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks",
          "url": "https://arxiv.org/abs/2005.11401"
        },
        {
          "title": "Karpukhin et al. (2020), Dense Passage Retrieval for Open-Domain Question Answering",
          "url": "https://arxiv.org/abs/2004.04906"
        },
        {
          "title": "Gao et al. (2023), Retrieval-Augmented Generation for Large Language Models: A Survey",
          "url": "https://arxiv.org/abs/2312.10997"
        },
        {
          "title": "Liu et al. (2023), Lost in the Middle: How Language Models Use Long Contexts",
          "url": "https://arxiv.org/abs/2307.03172"
        },
        {
          "title": "Malkov & Yashunin (2016), Efficient and robust approximate nearest neighbor search using HNSW graphs",
          "url": "https://arxiv.org/abs/1603.09320"
        },
        {
          "title": "Anthropic (2024), Introducing Contextual Retrieval",
          "url": "https://www.anthropic.com/news/contextual-retrieval"
        },
        {
          "title": "Es et al. (2023), RAGAS: Automated Evaluation of Retrieval Augmented Generation",
          "url": "https://arxiv.org/abs/2309.15217"
        },
        {
          "title": "Edge et al. (2024), From Local to Global: A Graph RAG Approach to Query-Focused Summarization",
          "url": "https://arxiv.org/abs/2404.16130"
        },
        {
          "title": "Gao et al. (2022), Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE)",
          "url": "https://arxiv.org/abs/2212.10496"
        },
        {
          "title": "Zheng et al. (2023), Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena",
          "url": "https://arxiv.org/abs/2306.05685"
        }
      ],
      "url": "https://aimap.page/en/learn/rag/what-is-rag/",
      "markdown": "## The problem RAG solves\n\nAn LLM's knowledge is frozen in its weights at training time ([What Is an LLM?](/learn/what-is-an-llm)). It does not know your company wiki, yesterday's prices or the contract on your desk — and when asked anyway, it may invent a plausible answer.\n\n**Retrieval-augmented generation (RAG)**, named in [Lewis et al. (2020)](https://arxiv.org/abs/2005.11401), fixes this with a simple idea: **look the information up first, then let the model answer using what was found.** It is an open-book exam instead of a closed-book one.\n\n## The pipeline at a glance\n\n```mermaid\nflowchart LR\n  subgraph Indexing[\"Offline: indexing\"]\n    D[\"Documents\"] --> C[\"Chunk\"] --> E[\"Embed\"] --> V[(\"Vector + keyword index\")]\n  end\n  subgraph Query[\"Online: answering\"]\n    Q[\"User question\"] --> QE[\"Embed query\"] --> R[\"Retrieve top-k (vector + BM25)\"]\n    V --> R\n    R --> RR[\"Re-rank\"] --> P[\"Prompt: question + passages + rules\"]\n    P --> L[\"LLM\"] --> A[\"Answer with citations\"]\n  end\n```\n\n### 1. Load and chunk\n\nExtract clean text from PDFs, web pages, docs and tickets, then split it into **chunks** of a few hundred tokens, often with some overlap. Chunks that are too large dilute relevance; chunks that are too small lose context. Splitting on headings and paragraphs beats splitting every N characters.\n\n### 2. Embed and index\n\nAn **embedding model** turns each chunk into a vector so that passages with similar meaning are close together ([Tokens and Embeddings](/learn/tokens-embeddings)). Dense retrievers of this kind ([Karpukhin et al., 2020](https://arxiv.org/abs/2004.04906)) outperform keyword search on many question-answering tasks. Vectors go into an index that supports fast **approximate nearest-neighbour** search, commonly HNSW graphs ([Malkov & Yashunin, 2016](https://arxiv.org/abs/1603.09320)) — in a vector database, Postgres with a vector extension, or simply in memory. Store metadata (source, date, permissions) alongside.\n\n### 3. Retrieve\n\nAt question time, embed the query and fetch the top-*k* most similar chunks. In practice **hybrid search** works best: combine vector similarity with classic keyword scoring (BM25), because embeddings are weak at exact names, codes and numbers while keywords miss paraphrases. Apply metadata filters, including **access control** — never retrieve what the user may not see.\n\n### 4. Re-rank\n\nA **re-ranker** (cross-encoder) reads the question and each candidate together and scores relevance much more accurately than vector distance, at higher cost — so it runs only on the top 20–100 candidates. Anthropic reported that adding context to chunks before embedding plus BM25 cut retrieval failures by 49%, and by 67% with re-ranking, on their test sets ([Contextual Retrieval](https://www.anthropic.com/news/contextual-retrieval)).\n\n### 5. Generate with citations\n\nBuild a prompt with the question, the selected passages (with IDs) and explicit rules: *answer only from the sources; cite the passage IDs; say \"I don't know\" if the sources are insufficient.* Put the most relevant passages first or last: models use information in the middle of long contexts less reliably ([Liu et al., 2023](https://arxiv.org/abs/2307.03172)).\n\n## RAG vs fine-tuning vs long context\n\n| | RAG | Fine-tuning | Long context only |\n|---|---|---|---|\n| Fresh / changing facts | ✅ update the index | ❌ retrain | ✅ but re-send every time |\n| Private data with permissions | ✅ filter per user | ❌ baked into weights | ⚠️ |\n| Citations | ✅ | ❌ | ⚠️ possible |\n| Change style / format | ⚠️ | ✅ | ⚠️ |\n| Cost per question | low | low | high for large corpora |\n\n## Evaluating a RAG system\n\nMeasure the two halves separately ([Gao et al., 2023](https://arxiv.org/abs/2312.10997) survey common metrics):\n\n- **Retrieval**: for a set of test questions with known relevant passages, how often is a correct passage in the top-*k* (recall@k)?\n- **Generation**: is the answer **faithful** to the retrieved passages, and does it actually answer the question? Frameworks such as RAGAS ([Es et al., 2023](https://arxiv.org/abs/2309.15217)) automate these checks with LLM judges — validate them against human labels.\n\nMost failures trace back to retrieval, so look at what was retrieved before tuning prompts.\n\n## Beyond basic RAG\n\n- **Agentic RAG** – an agent decides when and what to search, possibly several times ([How AI Agents Work](/learn/how-agents-work)); search becomes one of its tools ([Tool Calling and MCP](/learn/tool-calling-mcp)).\n- **GraphRAG** – builds a knowledge graph and community summaries to answer \"global\" questions about a whole corpus ([Edge et al., 2024](https://arxiv.org/abs/2404.16130)).\n\n## Try it without code\n\nDesktop apps such as [AnythingLLM](/apps/anythingllm) and [Cherry Studio](/apps/cherry-studio) let you drop in documents and chat with them using local or cloud models — a quick way to see chunking and retrieval in action, and their failure modes. Deeper dives: [Vector Databases](/learn/vector-databases) and [Chunking, Hybrid Search and Re-ranking](/learn/chunking-reranking) (in progress).\n\n## Algorithm cards\n\n| Algorithm | In one line | Original paper |\n|---|---|---|\n| Retrieval-augmented generation (RAG) | Retrieve relevant passages into the prompt, then generate from that evidence | [Lewis et al., 2020](https://arxiv.org/abs/2005.11401) |\n| HyDE | Have the model write a hypothetical answer and use its embedding to retrieve real documents | [Gao et al., 2022](https://arxiv.org/abs/2212.10496) |\n| GraphRAG | Extract an entity graph and community summaries to answer corpus-wide questions | [Edge et al., 2024](https://arxiv.org/abs/2404.16130) |\n| LLM-as-a-judge | A strong model scores answers against a rubric to evaluate quality and faithfulness | [Zheng et al., 2023](https://arxiv.org/abs/2306.05685) |\n\n## Must-read papers\n\n1. **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks** (2020) — where the name RAG and the jointly trained retriever + generator come from. [arXiv:2005.11401](https://arxiv.org/abs/2005.11401)\n2. **Precise Zero-Shot Dense Retrieval without Relevance Labels** (2022) — HyDE: rewrite the query with a generated hypothetical document when no labels exist. [arXiv:2212.10496](https://arxiv.org/abs/2212.10496)\n3. **From Local to Global: A Graph RAG Approach to Query-Focused Summarization** (2024) — GraphRAG: knowledge graphs and community summaries for “what is this corpus about” questions. [arXiv:2404.16130](https://arxiv.org/abs/2404.16130)\n\n## Hands-on project (about 8 hours)\n\nBuild RAG Q&A over a set of product docs (for example an open-source project’s official documentation). Prepare 30 test questions with reference answers and source passages; compare three chunking strategies (fixed length, by heading, semantic) and two retrieval methods (pure vector vs keyword + vector hybrid), log retrieval recall, faithfulness and answer relevance with [RAGAS](https://docs.ragas.io/en/stable/), and write a one-page conclusion. See the [LlamaIndex docs](https://developers.llamaindex.ai/python/framework/) and [Microsoft GraphRAG docs](https://microsoft.github.io/graphrag/).\n\n## Self-check\n\n<details><summary>1. What does RAG “faithfulness” measure?</summary>\n\nWhether every claim in the answer is supported by the retrieved passages. It does not measure correctness, only whether the answer strays from the sources.\n</details>\n\n<details><summary>2. What kind of question is GraphRAG good at?</summary>\n\nThematic, corpus-wide questions (e.g. “which risks do these reports keep raising?”) that chunk-by-chunk retrieval handles poorly.\n</details>\n\n<details><summary>3. What goes wrong with chunks that are too large or too small?</summary>\n\nToo large: unrelated text dilutes retrieval and wastes context. Too small: each chunk lacks context and the answer gets scattered across many chunks.\n</details>\n\n<details><summary>4. Why does HyDE improve retrieval?</summary>\n\nUser questions are short and phrased differently from documents; a generated hypothetical answer is closer to real documents in wording and embedding space, so relevant passages are easier to find.\n</details>"
    },
    "zh-hans": {
      "title": "RAG 详解：检索增强生成从零讲起",
      "description": "检索增强生成（RAG）是什么、为什么获取最新或私有知识时它比微调更合适，以及 RAG 流程的每一步：切分、向量化、向量检索、混合检索、重排、带引用的生成与评估。",
      "updated": "2026-10-22",
      "takeaways": [
        "RAG 在提问时从你自己的文档中检索相关段落放进提示，让模型依据证据回答，而不是凭记忆。",
        "流程分离线建索引（加载 → 切分 → 向量化 → 存储）和在线问答（检索 → 重排 → 带引用生成）两半。",
        "瓶颈在检索：切分方式、关键词加向量的混合检索、重排，往往比换哪个大模型更重要。",
        "检索和生成要分开评估：取回的段落对不对？回答是否忠实于这些段落？"
      ],
      "faq": [
        {
          "q": "该用 RAG 还是微调？",
          "a": "需要会变化的、私有的或者必须标注出处的事实时，用 RAG；想改变行为、格式或风格时，用微调。两者可以结合：微调过的模型照样可以用 RAG。"
        },
        {
          "q": "做 RAG 一定要向量数据库吗？",
          "a": "不一定。小规模资料放在内存里，或用带向量扩展的 SQLite、Postgres 就够了；当文本块达到数百万、需要大规模过滤或高并发时，专用向量数据库才更有优势。单用关键词检索（BM25）也是合格的检索器。"
        },
        {
          "q": "上下文窗口这么长了，还需要 RAG 吗？",
          "a": "长上下文有帮助，但把所有内容都塞进去又慢又贵，而且模型对长输入中间部分的信息利用得不够可靠。检索能让提示短小、便宜、聚焦，还能给出出处。"
        },
        {
          "q": "为什么我的 RAG 还是会胡说？",
          "a": "通常是因为检索漏掉了正确段落、文本块缺少上下文，或者提示里没有要求模型只依据资料回答、资料不足时明确说不知道。"
        }
      ],
      "sources": [
        {
          "title": "Lewis 等（2020），Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks",
          "url": "https://arxiv.org/abs/2005.11401"
        },
        {
          "title": "Karpukhin 等（2020），Dense Passage Retrieval for Open-Domain Question Answering",
          "url": "https://arxiv.org/abs/2004.04906"
        },
        {
          "title": "Gao 等（2023），Retrieval-Augmented Generation for Large Language Models: A Survey（同济大学等）",
          "url": "https://arxiv.org/abs/2312.10997"
        },
        {
          "title": "Liu 等（2023），Lost in the Middle: How Language Models Use Long Contexts",
          "url": "https://arxiv.org/abs/2307.03172"
        },
        {
          "title": "Malkov & Yashunin（2016），HNSW 近似最近邻图索引",
          "url": "https://arxiv.org/abs/1603.09320"
        },
        {
          "title": "Anthropic（2024），Introducing Contextual Retrieval",
          "url": "https://www.anthropic.com/news/contextual-retrieval"
        },
        {
          "title": "Es 等（2023），RAGAS: Automated Evaluation of Retrieval Augmented Generation",
          "url": "https://arxiv.org/abs/2309.15217"
        },
        {
          "title": "Edge 等（2024），From Local to Global: A Graph RAG Approach（GraphRAG）",
          "url": "https://arxiv.org/abs/2404.16130"
        },
        {
          "title": "Gao 等（2022），Precise Zero-Shot Dense Retrieval without Relevance Labels（HyDE）",
          "url": "https://arxiv.org/abs/2212.10496"
        },
        {
          "title": "Zheng 等（2023），Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena",
          "url": "https://arxiv.org/abs/2306.05685"
        }
      ],
      "url": "https://aimap.page/zh-hans/learn/rag/what-is-rag/",
      "markdown": "## RAG 要解决什么问题\n\n大模型的知识在训练时就被冻结在权重里了（见 [大模型是什么](/learn/what-is-an-llm)）。它不知道你公司的内部文档、昨天的价格、你桌上的合同；硬问它，它可能编一个听起来合理的答案。\n\n**检索增强生成（RAG）**这个名字来自 [Lewis 等（2020）](https://arxiv.org/abs/2005.11401)，思路很简单：**先把资料查出来，再让模型根据查到的内容回答。** 闭卷考试变成了开卷考试。\n\n## 流程总览\n\n```mermaid\nflowchart LR\n  subgraph Indexing[\"离线：建索引\"]\n    D[\"文档\"] --> C[\"切分\"] --> E[\"向量化\"] --> V[(\"向量 + 关键词索引\")]\n  end\n  subgraph Query[\"在线：回答\"]\n    Q[\"用户问题\"] --> QE[\"问题向量化\"] --> R[\"检索 top-k（向量 + BM25）\"]\n    V --> R\n    R --> RR[\"重排\"] --> P[\"提示：问题 + 段落 + 规则\"]\n    P --> L[\"大模型\"] --> A[\"带引用的回答\"]\n  end\n```\n\n### 1. 加载与切分\n\n从 PDF、网页、文档、工单里提取干净的文本，再切成几百个 token 的**文本块**，块与块之间常留一些重叠。块太大，相关性被稀释；块太小，丢了上下文。按标题和段落切，比每隔 N 个字符硬切效果好。\n\n### 2. 向量化与建索引\n\n**向量模型**把每个文本块变成一个向量，让意思相近的段落彼此靠近（见 [Token 与 Embedding](/learn/tokens-embeddings)）。这类稠密检索器（[Karpukhin 等，2020](https://arxiv.org/abs/2004.04906)）在很多问答任务上胜过关键词搜索。向量存进支持快速**近似最近邻**搜索的索引，常用 HNSW 图（[Malkov & Yashunin，2016](https://arxiv.org/abs/1603.09320)），可以放在向量数据库、带向量扩展的 Postgres，或者直接放内存。来源、日期、权限等元数据要一起存。\n\n### 3. 检索\n\n提问时，把问题向量化，取出最相似的 top-*k* 个块。实践中**混合检索**效果最好：把向量相似度和经典的关键词打分（BM25）结合起来，因为向量不擅长精确的名称、编号和数字，关键词又抓不住换了说法的同义表达。还要加元数据过滤，包括**权限控制**：用户无权看的内容绝不能检索出来。\n\n### 4. 重排\n\n**重排模型**（交叉编码器）把问题和每个候选段落放在一起读，相关性打分比向量距离准得多，但更贵，所以只对前 20 到 100 个候选运行。Anthropic 报告，在向量化之前给每个块补上上下文，再结合 BM25，在他们的测试集上把检索失败率降低了 49%，加上重排后降低了 67%（[Contextual Retrieval](https://www.anthropic.com/news/contextual-retrieval)）。\n\n### 5. 带引用生成\n\n把问题、选出的段落（带编号）和明确的规则一起写进提示：*只根据资料回答；标注段落编号；资料不够就说“不知道”。* 最相关的段落放在开头或结尾，因为模型对长上下文中间部分的利用不够可靠（[Liu 等，2023](https://arxiv.org/abs/2307.03172)）。\n\n## RAG、微调与长上下文对比\n\n| | RAG | 微调 | 只靠长上下文 |\n|---|---|---|---|\n| 最新、会变化的事实 | ✅ 更新索引即可 | ❌ 要重新训练 | ✅ 但每次都要重新发送 |\n| 带权限的私有数据 | ✅ 按用户过滤 | ❌ 写死在权重里 | ⚠️ |\n| 标注出处 | ✅ | ❌ | ⚠️ 可以做到 |\n| 改变风格、格式 | ⚠️ | ✅ | ⚠️ |\n| 每次提问成本 | 低 | 低 | 资料多时很高 |\n\n## 怎样评估 RAG\n\n两半分开测（常用指标见 [Gao 等，2023](https://arxiv.org/abs/2312.10997) 的综述）：\n\n- **检索**：准备一批已知相关段落的测试问题，看正确段落进入 top-*k* 的比例（recall@k）。\n- **生成**：回答是否**忠实**于检索到的段落，是否真正回答了问题？RAGAS（[Es 等，2023](https://arxiv.org/abs/2309.15217)）等框架用大模型当裁判自动做这些检查，但要先用人工标注校验裁判本身。\n\n多数失败都能追溯到检索环节，所以调提示词之前，先看看到底检索出了什么。\n\n## 进阶形态\n\n- **Agentic RAG**：由 agent 决定什么时候搜、搜什么，可能搜好几轮（见 [AI Agent 工作原理](/learn/how-agents-work)）；搜索变成它的一个工具（见 [工具调用与 MCP](/learn/tool-calling-mcp)）。\n- **GraphRAG**：先构建知识图谱和社区摘要，用来回答关于整个语料的“全局”问题（[Edge 等，2024](https://arxiv.org/abs/2404.16130)）。\n\n## 不写代码也能试\n\n[AnythingLLM](/apps/anythingllm)、[Cherry Studio](/apps/cherry-studio) 等桌面应用可以直接拖入文档，用本地或云端模型和文档对话，很快就能看到切分和检索是怎么工作的，也能看到它们会在哪里出错。深入阅读：[向量数据库与相似度搜索](/learn/vector-databases)、[切分、混合检索与重排](/learn/chunking-reranking)（编写中）。\n\n## 算法卡片\n\n| 算法 | 一句话 | 原始论文 |\n|---|---|---|\n| 检索增强生成 RAG | 先检索相关段落放进提示，再让模型依据证据生成 | [Lewis 等，2020](https://arxiv.org/abs/2005.11401) |\n| HyDE 假设文档检索 | 先让模型写一段“假想答案”，用它的向量去检索真实文档 | [Gao 等，2022](https://arxiv.org/abs/2212.10496) |\n| GraphRAG | 从语料抽取实体关系图和社区摘要，回答需要全局汇总的问题 | [Edge 等，2024](https://arxiv.org/abs/2404.16130) |\n| 大模型当评委 | 用强模型按评分标准打分，自动评估回答质量和忠实度 | [Zheng 等，2023](https://arxiv.org/abs/2306.05685) |\n\n## 必读论文\n\n1. **Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks**（2020）——RAG 这个名字和“检索器 + 生成器”联合训练的出处。[arXiv:2005.11401](https://arxiv.org/abs/2005.11401)\n2. **Precise Zero-Shot Dense Retrieval without Relevance Labels**（2022）——HyDE：没有标注数据时，用生成的假想文档改写查询。[arXiv:2212.10496](https://arxiv.org/abs/2212.10496)\n3. **From Local to Global: A Graph RAG Approach to Query-Focused Summarization**（2024）——GraphRAG：用知识图谱和社区摘要回答“整个语料讲了什么”。[arXiv:2404.16130](https://arxiv.org/abs/2404.16130)\n\n## 动手项目（约 8 学时）\n\n为一套产品文档（例如某个开源项目的官方文档）搭一个 RAG 问答：准备 30 个带标准答案和出处段落的测试问题；比较三种切分策略（固定长度、按标题、按语义）和两种检索方式（纯向量、关键词 + 向量混合），用 [RAGAS](https://docs.ragas.io/en/stable/) 记录检索召回率、忠实度和答案相关性，写一页结论。可参考 [LlamaIndex 文档](https://developers.llamaindex.ai/python/framework/) 和 [Microsoft GraphRAG 文档](https://microsoft.github.io/graphrag/)。\n\n## 自测\n\n<details><summary>1. RAG 的“忠实度”衡量什么？</summary>\n\n回答里的每个说法是否都能在检索到的段落里找到依据；它不衡量答案是否正确，只衡量有没有“脱离资料”。\n</details>\n\n<details><summary>2. GraphRAG 擅长哪类问题？</summary>\n\n需要综合整个语料的主题类、全局汇总类问题（如“这些报告反复提到哪些风险”），普通的按片段检索很难答好。\n</details>\n\n<details><summary>3. 切块太大或太小各有什么问题？</summary>\n\n太大：一块里混进很多无关内容，检索不准、占用上下文；太小：单块缺少上下文，答案被切散到多块里。\n</details>\n\n<details><summary>4. HyDE 为什么能提高检索效果？</summary>\n\n用户问题往往很短，与文档的写法差别大；先生成一段假想答案，它在措辞和向量空间里更接近真实文档，所以更容易检索到相关段落。\n</details>"
    }
  }
}