What Is a Large Language Model (LLM)? A Plain-Language Guide

The one-sentence answer

A large language model (LLM) is a computer program — specifically a very large neural network — that has been trained on an enormous amount of text to do one thing: given some text, predict what comes next. ChatGPT, Claude, Gemini, DeepSeek, Qwen and Llama are all built on this idea.

That sounds too simple to explain an assistant that can write code, summarise a contract or plan a trip. The surprise of the last few years is that, when a model is big enough and trained on enough text, getting very good at "what comes next" forces it to pick up grammar, facts, styles of reasoning and even some programming skill along the way.

Step 1: text becomes tokens

Models do not read letters or whole words. Text is first split into tokens — common words, pieces of words or punctuation. "Unbelievable!" might become Un believ able !. Each token is mapped to a number, and each number to a list of numbers (a vector) that the network can compute with. The next lesson, Tokens and Embeddings, explains this in detail.

Step 2: predict the next token, again and again

Given the tokens so far, the model outputs a probability for every token in its vocabulary. For "The capital of France is", the token " Paris" gets a very high probability. The program picks one token (usually one of the most likely), appends it, and repeats. A long answer is just this loop run hundreds or thousands of times.

flowchart LR
  A["Your prompt"] --> B["Split into tokens"]
  B --> C["Neural network (Transformer)"]
  C --> D["Probabilities for the next token"]
  D --> E["Pick one token"]
  E -->|"append and repeat"| C
  E --> F["Answer, token by token"]

The network architecture almost all LLMs use is the Transformer, introduced by Google researchers in 2017 (Vaswani et al.). Its key ingredient, attention, lets every token look at every earlier token to decide what matters. See Transformers and Attention.

Step 3: what "parameters" and "training" mean

Inside the network are parameters (also called weights): billions of numbers that decide how input tokens are transformed into predictions. Before training they are random, and the model outputs gibberish.

Training shows the model trillions of tokens of text — web pages, books, code, papers — and, at every position, compares its prediction with the token that actually came next. A procedure called gradient descent nudges every parameter slightly so the right token becomes a bit more likely next time (How Machines Learn explains this). Repeat this trillions of times on thousands of GPUs, and the parameters end up encoding a compressed statistical picture of the text the model saw.

Two practical consequences:

• The model's knowledge lives in its parameters, not in a searchable database. It cannot "look something up" unless you give it a tool, and it does not know about events after its training data was collected (its knowledge cut-off).
• Bigger generally means better — up to a point. Kaplan et al. (2020) found that prediction error falls smoothly as parameters, data and compute grow. GPT-3 (Brown et al., 2020) had 175 billion parameters and showed it could perform new tasks from just a few examples in the prompt.

Step 4: from text predictor to assistant

A model trained only to continue internet text (a base model) is not a good assistant: asked a question, it may continue with more questions. Chat assistants receive extra training:

1. Instruction tuning on examples of requests and good responses.
2. Reinforcement learning from human feedback (RLHF) or similar methods, where people (or other models) rank answers and the model is pushed toward the preferred ones. OpenAI's InstructGPT paper (Ouyang et al., 2022) showed that a much smaller model tuned this way was preferred by people over the raw 175B GPT-3.

The lesson How LLMs Learn covers these stages.

What LLMs are good at — and where they fail

Most weaknesses are addressed not by a bigger model but by giving the model tools: a search engine or document store (RAG), a code interpreter, or other software it can call (Tool Calling and MCP). A model that runs in a loop, choosing tools to reach a goal, is called an agent (How AI Agents Work).

Open and closed models

Some companies only offer their models through an API or app (closed models, e.g. GPT, Claude, Gemini). Others publish the trained parameters so anyone can run them (open-weight models, e.g. Llama, Qwen, DeepSeek, Mistral, Gemma). You can run open-weight models on a laptop with apps such as Ollama or LM Studio — a good way to see that an LLM really is "just" a large file of numbers plus a program that runs the prediction loop.

Try it yourself

1. Ask an assistant the same open-ended question three times. The answers differ because the next token is sampled from probabilities, not fixed.
2. Ask about a very recent event. Note whether it admits it does not know, or invents an answer.
3. Ask it to multiply two six-digit numbers, then check with a calculator.

These three experiments show the core facts of this lesson: LLMs predict probable text, have a knowledge cut-off and can be confidently wrong — and that is exactly why the rest of this curriculum exists.