// ∿ New Architectures: SSM & Linear Attention

Hybrid Architectures (Jamba)

Interleaves attention and Mamba layers (plus MoE) in one model.

混合架构（Transformer+SSM） 2024 Advanced core

IN PLAIN WORDS

A few attention layers handle precise recall while many Mamba layers process long sequences cheaply. Jamba was the first production-grade hybrid; hybrids are now a major trend.

Used in: AI21 Jamba models

TIMELINE · prerequisites → this → what builds on it

2017 Sparsely-Gate…2023 Mamba (Select…2024 Hybrid Architectu…

01Original paper

Jamba: A Hybrid Transformer-Mamba Language Model

Opher Lieber, Barak Lenz, Hofit Bata, et al. · 2024

arXiv ↗ ✓ Verified · arXiv API

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

Hybrid Architectures (Jamba)

Then you can learn

Nothing in the atlas builds on it yet

Related:Gated Delta Networks

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Official docs

huggingface.co ↗EN

04Where the course teaches it

U43 MoE & Beyond-Transformer Architectures Sparse experts, load balancing, Mamba/SSMs, linear attention and hybrid architectures, diffusion language models. 10 h18 algorithms5 papers

On learning paths:Generative Models & Frontier Research

For AI / data

Verification: verified, checked 2026-10-21.
JSON: /api/learn/algorithms.json (id: jamba)