// ◎ Transformers & Attention

Attention (Bahdanau)

Lets the decoder look back at the most relevant input positions.

注意力机制（Bahdanau） 2014 Foundational core

IN PLAIN WORDS

At each step the decoder scores all encoder positions and takes a weighted sum, instead of squeezing the sentence into one vector. It is the precursor of Transformer attention.

Used in: Neural machine translation

TIMELINE · prerequisites → this → what builds on it

2014 Sequence-to-S…2014 Attention (Bahdan…2017 Transformer &…

01Original paper

Neural Machine Translation by Jointly Learning to Align and Translate

Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio · 2014

arXiv ↗ ✓ Verified · arXiv API

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

Sequence-to-Sequence Learning

Attention (Bahdanau)

Then you can learn

Transformer & Self-Attention

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Explainers

04Where the course teaches it

U32 CNNs, RNNs and the Birth of Attention Convolutional networks, LSTMs, Seq2Seq, word vectors and Bahdanau attention. 10 h7 algorithms5 papers

On learning paths:Zero to Deep Learning

For AI / data

Verification: verified, checked 2026-10-21.
JSON: /api/learn/algorithms.json (id: bahdanau-attention)