// ◎ Transformers & Attention

T5 (Text-to-Text Transfer Transformer)

Casts every NLP task as text in, text out.

T5 文本到文本 2019 Foundational core

IN PLAIN WORDS

Translation, summarization and classification all become prefixed text tasks for one encoder–decoder. The paper is a systematic study of objectives, data and scale.

Used in: Flan-T5; text encoders for image models

TIMELINE · prerequisites → this → what builds on it

2017 Transformer &…2018 BERT2019 T5 (Text-to-Text …2021 Instruction T…

01Original paper

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Colin Raffel, Noam Shazeer, Adam Roberts, et al. · 2019

arXiv ↗ ✓ Verified · arXiv API

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

T5 (Text-to-Text Transfer Transformer)

Then you can learn

Instruction Tuning (FLAN)

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Courses

huggingface.co ↗MULTI

04Where the course teaches it

U41 Transformers & Attention Self-attention, multi-head attention, positional encoding, residuals and normalisation — build a GPT from scratch. 16 h6 algorithms3 paperslesson ready

For AI / data

Verification: verified, checked 2026-10-21.
JSON: /api/learn/algorithms.json (id: t5)