{
  "id": "what-is-an-llm",
  "module": "ai-basics",
  "level": 1,
  "minutes": 15,
  "audiences": [
    "beginner",
    "developer",
    "researcher"
  ],
  "software": [
    "chatgpt",
    "claude",
    "gemini",
    "ollama"
  ],
  "text": {
    "en": {
      "title": "What Is a Large Language Model (LLM)? A Plain-Language Guide",
      "description": "What a large language model is, how it predicts the next token, what parameters and training mean, and what LLMs can and cannot do — explained without maths, with sources.",
      "updated": "2026-10-10",
      "takeaways": [
        "An LLM is a neural network trained to predict the next token (a word piece) in text; everything else it does is built on that skill.",
        "Its “knowledge” is stored in billions of numeric parameters learned from huge text corpora, not in a database it can look up.",
        "Chat assistants are LLMs that received extra training (instruction tuning and human feedback) to follow instructions and be helpful.",
        "LLMs can be fluent and wrong: they have a knowledge cut-off, can hallucinate, and need tools or retrieval for fresh or exact facts."
      ],
      "faq": [
        {
          "q": "Is ChatGPT a large language model?",
          "a": "ChatGPT is a product built on top of large language models from OpenAI (the GPT family). The same is true of Claude (Anthropic), Gemini (Google) and many others: the chat app adds a user interface, safety layers and tools around the model."
        },
        {
          "q": "What does the “large” in large language model mean?",
          "a": "It refers to the number of learned parameters (from billions to over a trillion) and the amount of training text (trillions of tokens). Research on scaling laws showed that bigger models trained on more data predictably get better at predicting text."
        },
        {
          "q": "Does an LLM understand what it says?",
          "a": "Researchers disagree on the word “understand”. What is measurable is that LLMs build rich internal representations of concepts and can use them flexibly, but they also make errors no human expert would make, so their output should be checked."
        },
        {
          "q": "Can I run an LLM on my own computer?",
          "a": "Yes. Open-weight models such as Llama, Qwen, Mistral or Gemma can run locally with tools like Ollama, LM Studio or llama.cpp, provided you have enough memory for the model size you choose."
        }
      ],
      "sources": [
        {
          "title": "Vaswani et al. (2017), Attention Is All You Need",
          "url": "https://arxiv.org/abs/1706.03762"
        },
        {
          "title": "Brown et al. (2020), Language Models are Few-Shot Learners (GPT-3)",
          "url": "https://arxiv.org/abs/2005.14165"
        },
        {
          "title": "Kaplan et al. (2020), Scaling Laws for Neural Language Models",
          "url": "https://arxiv.org/abs/2001.08361"
        },
        {
          "title": "Ouyang et al. (2022), Training language models to follow instructions with human feedback",
          "url": "https://arxiv.org/abs/2203.02155"
        },
        {
          "title": "Zhao et al. (2023), A Survey of Large Language Models",
          "url": "https://arxiv.org/abs/2303.18223"
        },
        {
          "title": "3Blue1Brown, Large Language Models explained briefly (video)",
          "url": "https://www.3blue1brown.com/lessons/mini-llm"
        }
      ],
      "url": "https://aimap.page/en/learn/ai-basics/what-is-an-llm/",
      "markdown": "## The one-sentence answer\n\nA **large language model (LLM)** is a computer program — specifically a very large neural network — that has been trained on an enormous amount of text to do one thing: **given some text, predict what comes next.** ChatGPT, Claude, Gemini, DeepSeek, Qwen and Llama are all built on this idea.\n\nThat sounds too simple to explain an assistant that can write code, summarise a contract or plan a trip. The surprise of the last few years is that, when a model is big enough and trained on enough text, getting very good at \"what comes next\" forces it to pick up grammar, facts, styles of reasoning and even some programming skill along the way.\n\n## Step 1: text becomes tokens\n\nModels do not read letters or whole words. Text is first split into **tokens** — common words, pieces of words or punctuation. \"Unbelievable!\" might become `Un` `believ` `able` `!`. Each token is mapped to a number, and each number to a list of numbers (a vector) that the network can compute with. The next lesson, [Tokens and Embeddings](/learn/tokens-embeddings), explains this in detail.\n\n## Step 2: predict the next token, again and again\n\nGiven the tokens so far, the model outputs a **probability for every token in its vocabulary**. For \"The capital of France is\", the token \" Paris\" gets a very high probability. The program picks one token (usually one of the most likely), appends it, and repeats. A long answer is just this loop run hundreds or thousands of times.\n\n```mermaid\nflowchart LR\n  A[\"Your prompt\"] --> B[\"Split into tokens\"]\n  B --> C[\"Neural network (Transformer)\"]\n  C --> D[\"Probabilities for the next token\"]\n  D --> E[\"Pick one token\"]\n  E -->|\"append and repeat\"| C\n  E --> F[\"Answer, token by token\"]\n```\n\nThe network architecture almost all LLMs use is the **Transformer**, introduced by Google researchers in 2017 ([Vaswani et al.](https://arxiv.org/abs/1706.03762)). Its key ingredient, *attention*, lets every token look at every earlier token to decide what matters. See [Transformers and Attention](/learn/transformer-attention).\n\n## Step 3: what \"parameters\" and \"training\" mean\n\nInside the network are **parameters** (also called weights): billions of numbers that decide how input tokens are transformed into predictions. Before training they are random, and the model outputs gibberish.\n\n**Training** shows the model trillions of tokens of text — web pages, books, code, papers — and, at every position, compares its prediction with the token that actually came next. A procedure called gradient descent nudges every parameter slightly so the right token becomes a bit more likely next time ([How Machines Learn](/learn/ml-basics) explains this). Repeat this trillions of times on thousands of GPUs, and the parameters end up encoding a compressed statistical picture of the text the model saw.\n\nTwo practical consequences:\n\n- **The model's knowledge lives in its parameters**, not in a searchable database. It cannot \"look something up\" unless you give it a tool, and it does not know about events after its training data was collected (its *knowledge cut-off*).\n- **Bigger generally means better** — up to a point. [Kaplan et al. (2020)](https://arxiv.org/abs/2001.08361) found that prediction error falls smoothly as parameters, data and compute grow. GPT-3 ([Brown et al., 2020](https://arxiv.org/abs/2005.14165)) had 175 billion parameters and showed it could perform new tasks from just a few examples in the prompt.\n\n## Step 4: from text predictor to assistant\n\nA model trained only to continue internet text (a **base model**) is not a good assistant: asked a question, it may continue with more questions. Chat assistants receive extra training:\n\n1. **Instruction tuning** on examples of requests and good responses.\n2. **Reinforcement learning from human feedback (RLHF)** or similar methods, where people (or other models) rank answers and the model is pushed toward the preferred ones. OpenAI's InstructGPT paper ([Ouyang et al., 2022](https://arxiv.org/abs/2203.02155)) showed that a much smaller model tuned this way was preferred by people over the raw 175B GPT-3.\n\nThe lesson [How LLMs Learn](/learn/how-llms-learn) covers these stages.\n\n## What LLMs are good at — and where they fail\n\n| Good at | Weak at |\n|---|---|\n| Writing, rewriting, translating, summarising | Facts after the knowledge cut-off |\n| Explaining concepts at any level | Exact arithmetic and counting without tools |\n| Writing and reviewing code | Admitting uncertainty (they can **hallucinate**) |\n| Extracting structure from messy text | Long multi-step tasks without checks |\n| Following examples given in the prompt | Knowing the source of what they \"remember\" |\n\nMost weaknesses are addressed not by a bigger model but by **giving the model tools**: a search engine or document store ([RAG](/learn/what-is-rag)), a code interpreter, or other software it can call ([Tool Calling and MCP](/learn/tool-calling-mcp)). A model that runs in a loop, choosing tools to reach a goal, is called an **agent** ([How AI Agents Work](/learn/how-agents-work)).\n\n## Open and closed models\n\nSome companies only offer their models through an API or app (*closed* models, e.g. GPT, Claude, Gemini). Others publish the trained parameters so anyone can run them (*open-weight* models, e.g. Llama, Qwen, DeepSeek, Mistral, Gemma). You can run open-weight models on a laptop with apps such as [Ollama](/apps/ollama) or [LM Studio](/apps/lm-studio) — a good way to see that an LLM really is \"just\" a large file of numbers plus a program that runs the prediction loop.\n\n## Try it yourself\n\n1. Ask an assistant the same open-ended question three times. The answers differ because the next token is *sampled* from probabilities, not fixed.\n2. Ask about a very recent event. Note whether it admits it does not know, or invents an answer.\n3. Ask it to multiply two six-digit numbers, then check with a calculator.\n\nThese three experiments show the core facts of this lesson: LLMs predict probable text, have a knowledge cut-off and can be confidently wrong — and that is exactly why the rest of this curriculum exists."
    },
    "zh-hans": {
      "title": "大模型是什么？一文看懂大语言模型（LLM）",
      "description": "用大白话讲清大语言模型：它如何预测下一个词、参数和训练是什么意思、能做什么不能做什么。不需要数学基础，附权威出处。",
      "updated": "2026-10-10",
      "takeaways": [
        "大语言模型（LLM）是一个被训练来“预测下一个 token（词片段）”的神经网络，它的其他本事都建立在这个能力之上。",
        "它的“知识”存放在数十亿乃至上万亿个参数里，是从海量文本中学来的，而不是一个可以查询的数据库。",
        "聊天助手是在基座模型之上又做了指令微调和人类反馈训练的大模型，所以会听指令、会对话。",
        "大模型可能说得很流畅却是错的：它有知识截止日期，会“幻觉”，需要检索或工具来获取最新、精确的事实。"
      ],
      "faq": [
        {
          "q": "ChatGPT 是大模型吗？",
          "a": "ChatGPT 是建立在 OpenAI 大模型（GPT 系列）之上的产品。Claude（Anthropic）、Gemini（Google）、豆包、Kimi 等也一样：聊天应用在模型外面加了界面、安全措施和工具。"
        },
        {
          "q": "大模型的“大”指什么？",
          "a": "指参数数量（从几十亿到上万亿）和训练文本的规模（数万亿个 token）。缩放定律研究发现，模型越大、数据越多，预测文本的能力就会按可预测的规律变好。"
        },
        {
          "q": "大模型真的“理解”它说的话吗？",
          "a": "“理解”这个词学界有争议。可以测量的事实是：大模型内部形成了丰富的概念表征并能灵活运用，但它也会犯人类专家不会犯的错，所以输出需要核对。"
        },
        {
          "q": "可以在自己电脑上运行大模型吗？",
          "a": "可以。Llama、Qwen、Mistral、Gemma 等开放权重模型，可以用 Ollama、LM Studio、llama.cpp 在本地运行，前提是内存够放下你选的模型。"
        }
      ],
      "sources": [
        {
          "title": "Vaswani 等（2017），Attention Is All You Need",
          "url": "https://arxiv.org/abs/1706.03762"
        },
        {
          "title": "Brown 等（2020），Language Models are Few-Shot Learners（GPT-3）",
          "url": "https://arxiv.org/abs/2005.14165"
        },
        {
          "title": "Kaplan 等（2020），Scaling Laws for Neural Language Models",
          "url": "https://arxiv.org/abs/2001.08361"
        },
        {
          "title": "Ouyang 等（2022），Training language models to follow instructions with human feedback（InstructGPT）",
          "url": "https://arxiv.org/abs/2203.02155"
        },
        {
          "title": "Zhao 等（2023），A Survey of Large Language Models（中国人民大学，大模型综述）",
          "url": "https://arxiv.org/abs/2303.18223"
        },
        {
          "title": "3Blue1Brown，Large Language Models explained briefly（视频）",
          "url": "https://www.3blue1brown.com/lessons/mini-llm"
        }
      ],
      "url": "https://aimap.page/zh-hans/learn/ai-basics/what-is-an-llm/",
      "markdown": "## 一句话回答\n\n**大语言模型（Large Language Model，LLM）**，就是一个用海量文本训练出来的超大神经网络，它只被训练做一件事：**给定一段文字，预测接下来是什么。** ChatGPT、Claude、Gemini、DeepSeek、通义千问、Llama 都建立在这个思路上。\n\n这听起来太简单，不像能解释一个会写代码、会总结合同、会规划行程的助手。过去几年最大的意外就是：当模型足够大、训练文本足够多时，为了把“下一个是什么”猜得足够准，它不得不顺带学会语法、事实、推理方式，甚至编程。\n\n## 第一步：文字变成 token\n\n模型不是一个字一个字、也不是一个词一个词地读。文本先被切成 **token**：常见词、词的一部分或标点。英文 “Unbelievable!” 可能被切成 `Un` `believ` `able` `!`；中文里一个 token 常常对应一到两个汉字。每个 token 对应一个编号，每个编号再对应一串数字（向量），网络就拿这些向量做计算。下一课 [Token 与 Embedding](/learn/tokens-embeddings) 会详细讲。\n\n## 第二步：一遍又一遍地预测下一个 token\n\n根据已有的 token，模型会给词表里**每一个 token 算出一个概率**。对“法国的首都是”，“巴黎”的概率非常高。程序选出一个 token（通常是最可能的几个之一），接在后面，再重复。一段长回答，就是这个循环跑了成百上千次。\n\n```mermaid\nflowchart LR\n  A[\"你的提问\"] --> B[\"切分成 token\"]\n  B --> C[\"神经网络（Transformer）\"]\n  C --> D[\"下一个 token 的概率\"]\n  D --> E[\"选出一个 token\"]\n  E -->|\"接上去，再来一轮\"| C\n  E --> F[\"逐个 token 生成回答\"]\n```\n\n几乎所有大模型用的网络结构都是 **Transformer**，由 Google 研究者在 2017 年提出（[Vaswani 等](https://arxiv.org/abs/1706.03762)）。它的核心是“注意力”：每个 token 都能回头看前面所有 token，决定哪些更重要。详见 [Transformer 与注意力机制](/learn/transformer-attention)。\n\n## 第三步：“参数”和“训练”是什么意思\n\n网络里有大量**参数**（也叫权重）：数十亿个数字，决定了输入的 token 如何一步步变成预测。训练前它们是随机的，模型只会输出乱码。\n\n**训练**就是给模型看数万亿个 token 的文本（网页、书籍、代码、论文），在每个位置把它的预测和真正的下一个 token 比较，再用“梯度下降”把每个参数都微调一点点，让正确答案下次的概率高一点（见 [机器是怎么“学习”的](/learn/ml-basics)）。在成千上万块 GPU 上重复数万亿次，参数里就压缩存下了它见过的文本的统计规律。\n\n由此带来两个实际后果：\n\n- **模型的知识存在参数里**，不是一个可以检索的数据库。除非给它工具，否则它没法“去查一下”；训练数据收集之后发生的事，它也不知道（这就是*知识截止日期*）。\n- **一般来说越大越强**，但有上限。[Kaplan 等（2020）](https://arxiv.org/abs/2001.08361) 发现，随着参数、数据和算力增长，预测误差会平滑下降。GPT-3（[Brown 等，2020](https://arxiv.org/abs/2005.14165)）有 1750 亿参数，并证明只要在提示里给几个例子，它就能完成新任务。\n\n## 第四步：从“续写机器”到“助手”\n\n只学过续写网络文本的模型叫**基座模型**，它并不是好助手：你问它一个问题，它可能接着写出更多问题。聊天助手还要经过额外训练：\n\n1. **指令微调**：用“请求 + 优质回答”的示例继续训练。\n2. **基于人类反馈的强化学习（RLHF）**或类似方法：由人（或其他模型）给多个回答排序，把模型往更受欢迎的方向推。OpenAI 的 InstructGPT 论文（[Ouyang 等，2022](https://arxiv.org/abs/2203.02155)）显示，这样调教过的小得多的模型，比原始的 1750 亿参数 GPT-3 更受人们青睐。\n\n这些阶段在 [大模型如何学习](/learn/how-llms-learn) 一课中详细展开。\n\n## 大模型擅长什么、在哪里会失败\n\n| 擅长 | 不擅长 |\n|---|---|\n| 写作、改写、翻译、总结 | 知识截止日期之后的事实 |\n| 用不同深度解释概念 | 不借助工具的精确计算和计数 |\n| 写代码、审代码 | 承认自己不知道（会**幻觉**） |\n| 从杂乱文本中提取结构 | 没有检查环节的长链条任务 |\n| 模仿提示里给的示例 | 说清自己“记得”的内容出自哪里 |\n\n大多数短板靠的不是更大的模型，而是**给模型配工具**：搜索引擎或文档库（[RAG](/learn/what-is-rag)）、代码解释器，或者其他可以调用的软件（[工具调用与 MCP](/learn/tool-calling-mcp)）。在循环中自己挑选工具、一步步完成目标的模型，就叫 **Agent（智能体）**（[AI Agent 工作原理](/learn/how-agents-work)）。\n\n## 开源与闭源模型\n\n有些公司只通过 API 或应用提供模型（*闭源*，如 GPT、Claude、Gemini）；另一些公开训练好的参数，任何人都能运行（*开放权重*，如 Llama、通义千问、DeepSeek、Mistral、Gemma）。用 [Ollama](/apps/ollama) 或 [LM Studio](/apps/lm-studio) 就能在笔记本上跑开放权重模型，你会直观地看到：大模型“不过是”一个装满数字的大文件，加上一个跑预测循环的程序。\n\n## 动手试试\n\n1. 把同一个开放式问题问三遍。答案不一样，因为下一个 token 是按概率*采样*出来的，而不是固定的。\n2. 问一件刚发生的事，看它是承认不知道，还是编一个答案。\n3. 让它算两个六位数相乘，再用计算器核对。\n\n这三个小实验对应本课的核心事实：大模型预测的是“可能的文本”，有知识截止日期，也会自信地出错。这正是后面整套课程要解决的问题。"
    }
  }
}