// ⬡ Deep Learning Foundations

Muon Optimizer

An optimizer that orthogonalizes momentum updates for matrix parameters.

Muon 优化器 2024 Advanced core

IN PLAIN WORDS

For weight matrices, take the momentum and approximately orthogonalize it with Newton–Schulz iterations so all directions update evenly. Proposed by Keller Jordan in 2024 nanoGPT speedruns; Moonshot showed it scales in 2025.

Used in: Moonshot Kimi training (MuonClip); i14 Laplace 3B trains 2D weights with Muon and embeddings/norms with AdamW (AWS blog, Oct 2026); TorchTitan supports distributed Muon

TIMELINE · prerequisites → this → what builds on it

2017 AdamW (Decoup…2024 Muon Optimizer

01Original paper

Muon: An optimizer for hidden layers in neural networks (blog)

Keller Jordan · Blog · 2024

Open ↗ ✓ Verified · 官方页面 HTTP/WebFetch 核实

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

AdamW (Decoupled Weight Decay)

Muon Optimizer

Then you can learn

Nothing in the atlas builds on it yet

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Explainers

Reference code

github.com ↗EN

04Where the course teaches it

U31 Training Deep Nets SGD/momentum/Adam/AdamW/Muon, learning-rate schedules, BatchNorm/LayerNorm/RMSNorm, dropout and residuals. 10 h10 algorithms4 papers

For AI / data

Verification: verified, checked 2026-10-22.
JSON: /api/learn/algorithms.json (id: muon)