Deep Learning
From CNNs and RNNs to Transformers
A short history of architectures and why attention replaced recurrence.
Intermediate 15 min For: Beginners / Researchers ✎ In progress
✎ This lesson is still being written. Below is the planned outline; it is not indexed by search engines yet.
What this lesson will cover
- CNNs and images
- RNNs/LSTMs and sequences
- Sequence-to-sequence and early attention
- Parallelism: why Transformers scaled