// ⬡ Deep Learning Foundations
ELMo (Contextual Word Embeddings)
Bidirectional LSTM language-model features: the same word gets different vectors in different contexts.
ELMo 上下文词向量 2018 Foundational extended
IN PLAIN WORDS
Word2Vec gives each word one fixed vector, whatever the context. ELMo pre-trains a bidirectional LSTM language model and uses a learned mix of its layers as the word’s representation, so meaning shifts with context. It proved the power of pre-train-then-transfer, and BERT took over months later.
Used in: Contextual features behind several 2018 NLP benchmark gains; the key pre-BERT method
TIMELINE · prerequisites → this → what builds on it
01Original paper
Deep contextualized word representations
Matthew E. Peters, Mark Neumann, Mohit Iyyer, et al. · 2018
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
Then you can learn
- Nothing in the atlas builds on it yet
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
Courses
Explainers
Reference code
04Where the course teaches it
For AI / data
- Verification: verified, checked 2026-10-22.
- JSON: /api/learn/algorithms.json (id:
elmo)