// ◎ Transformers & Attention

GPT-3 & In-Context Learning

Large enough language models learn new tasks from a few examples in the prompt.

GPT-3 与上下文学习 2020 Foundational core

IN PLAIN WORDS

Trained only to predict the next token, GPT-3 performs new tasks from a few in-prompt examples without weight updates — making prompting a new kind of programming.

Used in: Prompting of every general-purpose chat model

TIMELINE · prerequisites → this → what builds on it

2017 Transformer &…2020 GPT-3 & In-Contex…2020 Scaling Laws2020 Retrieval-Aug…2021 Instruction T…2022 Chain-of-Thou…2023 Toolformer

01Original paper

Language Models are Few-Shot Learners

Tom B. Brown, Benjamin Mann, Nick Ryder, et al. · 2020

arXiv ↗ ✓ Verified · arXiv API

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

Transformer & Self-Attention

GPT-3 & In-Context Learning

Then you can learn

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Explainers

Reference code

github.com ↗EN

04Where the course teaches it

U00 What AI, ML, Deep Learning and LLMs Are One map of the AI family, and how an LLM “predicts the next token”. 3 h2 algorithms1 paperslesson ready U01 Prompting & Using AI Well Prompting techniques: role, context, examples, step-by-step thinking and structured output. 3 h3 algorithms1 papers U41 Transformers & Attention Self-attention, multi-head attention, positional encoding, residuals and normalisation — build a GPT from scratch. 16 h6 algorithms3 paperslesson ready

On learning paths:Understanding an LLM End to End

For AI / data

Verification: verified, checked 2026-10-21.
JSON: /api/learn/algorithms.json (id: gpt3-icl)