ESSENTIAL PAPERS · 150 · 1986 → 2025
必读经典
建议按这个顺序读：入门必读 → 核心 → 进阶。引用数来自 Semantic Scholar。
学习路线
大模型路线 (47)
- Learning representations by back-propagating errors 1986
- Long Short-Term Memory 1997
- Gradient-based learning applied to document recognition 1998
- ImageNet Classification with Deep Convolutional Neural Networks 2012
- Efficient Estimation of Word Representations in Vector Space 2013
- Adam: A Method for Stochastic Optimization 2014
- Neural Machine Translation by Jointly Learning to Align and Translate 2014
- Sequence to Sequence Learning with Neural Networks 2014
- Deep Residual Learning for Image Recognition 2015
- Attention Is All You Need 2017
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 2018
- Improving Language Understanding by Generative Pre-Training 2018
- Language Models are Few-Shot Learners 2020
- Scaling Laws for Neural Language Models 2020
- LLaMA: Open and Efficient Foundation Language Models 2023
- Training language models to follow instructions with human feedback 2022
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model 2023
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models 2022
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning 2025
- Auto-Encoding Variational Bayes 2013
- Dropout: A Simple Way to Prevent Neural Networks from Overfitting 2014
- Generative Adversarial Networks 2014
- Very Deep Convolutional Networks for Large-Scale Image Recognition 2014
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift 2015
- Distilling the Knowledge in a Neural Network 2015
- Layer Normalization 2016
- Deep contextualized word representations 2018
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer 2019
- Language Models are Unsupervised Multitask Learners 2019
- RoFormer: Enhanced Transformer with Rotary Position Embedding 2021
- PaLM: Scaling Language Modeling with Pathways 2022
- Training Compute-Optimal Large Language Models 2022
- GPT-4 Technical Report 2023
- Llama 2: Open Foundation and Fine-Tuned Chat Models 2023
- DeepSeek-V3 Technical Report 2024
- The Llama 3 Herd of Models 2024
- Deep reinforcement learning from human preferences 2017
- Learning to summarize from human feedback 2020
- Finetuned Language Models Are Zero-Shot Learners 2021
- Constitutional AI: Harmlessness from AI Feedback 2022
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback 2022
- Training Verifiers to Solve Math Word Problems 2021
- Large Language Models are Zero-Shot Reasoners 2022
- Self-Consistency Improves Chain of Thought Reasoning in Language Models 2022
- Let's Verify Step by Step 2023
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models 2024
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters 2024
智能体路线 (25)
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models 2022
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning 2025
- ReAct: Synergizing Reasoning and Acting in Language Models 2022
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues? 2023
- Training Verifiers to Solve Math Word Problems 2021
- Large Language Models are Zero-Shot Reasoners 2022
- Self-Consistency Improves Chain of Thought Reasoning in Language Models 2022
- Let's Verify Step by Step 2023
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models 2024
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters 2024
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks 2020
- Evaluating Large Language Models Trained on Code 2021
- Generative Agents: Interactive Simulacra of Human Behavior 2023
- Reflexion: Language Agents with Verbal Reinforcement Learning 2023
- Toolformer: Language Models Can Teach Themselves to Use Tools 2023
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering 2024
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models 2023
- OpenAI o1 System Card 2024
- Solving olympiad geometry without human demonstrations 2024
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale 2025
- Kimi k1.5: Scaling Reinforcement Learning with LLMs 2025
- s1: Simple test-time scaling 2025
- Competition-Level Code Generation with AlphaCode 2022
- WebArena: A Realistic Web Environment for Building Autonomous Agents 2023
- OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments 2024
视觉与生成路线 (17)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale 2020
- Learning Transferable Visual Models From Natural Language Supervision 2021
- Visual Instruction Tuning 2023
- Denoising Diffusion Probabilistic Models 2020
- High-Resolution Image Synthesis with Latent Diffusion Models 2021
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks 2015
- U-Net: Convolutional Networks for Biomedical Image Segmentation 2015
- You Only Look Once: Unified, Real-Time Object Detection 2015
- Flamingo: a Visual Language Model for Few-Shot Learning 2022
- Robust Speech Recognition via Large-Scale Weak Supervision 2022
- Segment Anything 2023
- Score-Based Generative Modeling through Stochastic Differential Equations 2020
- Zero-Shot Text-to-Image Generation 2021
- Classifier-Free Diffusion Guidance 2022
- Scalable Diffusion Models with Transformers 2022
- Adding Conditional Control to Text-to-Image Diffusion Models 2023
- Video generation models as world simulators 2024
入门必读 35
-
1986Nature❝32,740
Learning representations by back-propagating errors
反向传播的经典论文，至今所有神经网络训练的基础。
David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams
深度学习基础算法卡片 →
-
1997Neural Computation❝111,839
Long Short-Term Memory
LSTM 解决长序列梯度消失，Transformer 之前序列建模的主力。
Sepp Hochreiter, Jürgen Schmidhuber
深度学习基础算法卡片 →
-
1998Proceedings of the IEEE❝63,349
Gradient-based learning applied to document recognition
LeNet 与卷积网络的系统化论文，奠定了 CNN 做图像识别的范式。
Y. Lecun, L. Bottou, Y. Bengio +1
深度学习基础算法卡片 →
-
2012
ImageNet Classification with Deep Convolutional Neural Networks
AlexNet 在 ImageNet 上大幅领先，引爆了深度学习浪潮。
深度学习基础ⓘ
-
2013International Conference on Learning Representations❝35,307
Efficient Estimation of Word Representations in Vector Space
word2vec：把词变成向量，开启了“表示学习”在 NLP 中的普及。
Tomas Mikolov, Kai Chen, Greg Corrado +1
深度学习基础算法卡片 →
-
2014International Conference on Learning Representations❝172,629
Adam: A Method for Stochastic Optimization
Adam 优化器，深度学习默认的训练算法之一。
Diederik P. Kingma, Jimmy Ba
深度学习基础算法卡片 →
-
2014International Conference on Learning Representations❝30,227
Neural Machine Translation by Jointly Learning to Align and Translate
首次在神经机器翻译中引入注意力机制，是 Transformer 的直接前身。
Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio
深度学习基础算法卡片 →
-
2014Neural Information Processing Systems❝22,504
Sequence to Sequence Learning with Neural Networks
Seq2Seq 用编码器-解码器做机器翻译，确立了生成式序列建模框架。
Ilya Sutskever, Oriol Vinyals, Quoc V. Le
深度学习基础算法卡片 →
-
2015Computer Vision and Pattern Recognition❝242,528
Deep Residual Learning for Image Recognition
ResNet 的残差连接让上百层网络可训练，残差思想也进入了 Transformer。
Kaiming He, Xiangyu Zhang, Shaoqing Ren +1
深度学习基础算法卡片 →
-
2017Neural Information Processing Systems❝197,519
Attention Is All You Need
Transformer 原始论文，今天几乎所有大模型的架构基础。
Ashish Vaswani, Noam Shazeer, Niki Parmar +5
Transformer 与大模型预训练算法卡片 →
-
2018North American Chapter of the Association for Computational Linguistics❝122,956
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BERT 用双向掩码预训练刷新多项 NLP 基准，开启“预训练模型时代”。
Jacob Devlin, Ming-Wei Chang, Kenton Lee +1
Transformer 与大模型预训练算法卡片 →
-
2018
Improving Language Understanding by Generative Pre-Training
GPT-1：生成式预训练 + 微调的范式起点。
Transformer 与大模型预训练ⓘ
-
2020Neural Information Processing Systems❝64,882
Language Models are Few-Shot Learners
GPT-3 展示上下文学习（few-shot），让“提示”成为使用模型的新方式。
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Transformer 与大模型预训练算法卡片 →
-
2020arXiv.org❝9,364
Scaling Laws for Neural Language Models
缩放定律：模型性能随参数、数据、算力呈幂律提升，指导了大模型投入。
Jared Kaplan, Sam McCandlish, Tom Henighan +7
Transformer 与大模型预训练算法卡片 →
-
2023arXiv.org❝22,077
LLaMA: Open and Efficient Foundation Language Models
LLaMA 证明用公开数据训练的中等规模模型可媲美大模型，带动开源大模型生态。
Hugo Touvron, Thibaut Lavril, Gautier Izacard +11
Transformer 与大模型预训练
-
2022Neural Information Processing Systems❝25,131
Training language models to follow instructions with human feedback
InstructGPT：SFT + 奖励模型 + PPO 的三步法，ChatGPT 的技术原型。
Long Ouyang, Jeff Wu, Xu Jiang +17
指令微调与对齐算法卡片 →
-
2023Neural Information Processing Systems❝11,164
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
DPO 不需要单独的奖励模型和 RL，直接用偏好数据优化，成为主流对齐方法。
Rafael Rafailov, Archit Sharma, Eric Mitchell +3
指令微调与对齐算法卡片 →
-
2022Neural Information Processing Systems❝23,021
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
思维链提示：让模型先写推理步骤再作答，显著提升推理。
Jason Wei, Xuezhi Wang, Dale Schuurmans +6
推理与推理模型算法卡片 →
-
2025Nature❝5,905
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-R1：纯强化学习就能激发长链推理，并开源模型和蒸馏版本。
DeepSeek-AI, Daya Guo, Dejian Yang +197
推理与推理模型算法卡片 →
-
2022International Conference on Learning Representations❝12,767
ReAct: Synergizing Reasoning and Acting in Language Models
ReAct 让模型交替“思考-行动-观察”，是 Agent 的基本范式。
Shunyu Yao, Jeffrey Zhao, Dian Yu +4
智能体、工具与代码算法卡片 →
-
2023International Conference on Learning Representations❝4,445
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
SWE-bench 用真实 GitHub 问题评测模型修代码，成为编程 Agent 核心基准。
Carlos E. Jimenez, John Yang, Alexander Wettig +4
智能体、工具与代码
-
2020International Conference on Learning Representations❝71,916
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
ViT 把图像切块当 token 输入 Transformer，视觉进入 Transformer 时代。
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov +9
视觉与多模态算法卡片 →
-
2021International Conference on Machine Learning❝58,042
Learning Transferable Visual Models From Natural Language Supervision
CLIP 用 4 亿图文对做对比学习，图文对齐成为多模态基础。
Alec Radford, Jong Wook Kim, Chris Hallacy +9
视觉与多模态算法卡片 →
-
2023Neural Information Processing Systems❝11,569
Visual Instruction Tuning
LLaVA 视觉指令微调，开源多模态大模型的标准做法。
Haotian Liu, Chunyuan Li, Qingyang Wu +1
视觉与多模态算法卡片 →
-
2020Neural Information Processing Systems❝36,562
Denoising Diffusion Probabilistic Models
DDPM 让扩散模型真正可用，开启扩散生成时代。
Jonathan Ho, Ajay Jain, Pieter Abbeel
生成模型（图像/视频）算法卡片 →
-
2021Computer Vision and Pattern Recognition❝28,282
High-Resolution Image Synthesis with Latent Diffusion Models
潜扩散（Stable Diffusion 的基础）：在压缩潜空间里扩散，大幅降低成本。
Robin Rombach, Andreas Blattmann, Dominik Lorenz +2
生成模型（图像/视频）算法卡片 →
-
2021International Conference on Learning Representations❝24,527
LoRA: Low-Rank Adaptation of Large Language Models
LoRA 低秩微调，只训练很少参数就能适配大模型。
Edward J. Hu, Yelong Shen, Phillip Wallis +5
高效训练与推理算法卡片 →
-
2023Symposium on Operating Systems Principles❝9,252
Efficient Memory Management for Large Language Model Serving with PagedAttention
vLLM / PagedAttention 像操作系统分页一样管理 KV 缓存，大幅提高推理吞吐。
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang +6
高效训练与推理算法卡片 →
-
2013arXiv.org❝14,083
Playing Atari with Deep Reinforcement Learning
DQN 用深度网络从像素学会玩 Atari，深度强化学习的起点。
Volodymyr Mnih, Koray Kavukcuoglu, David Silver +4
强化学习、世界模型与机器人算法卡片 →
-
2017arXiv.org❝32,368
Proximal Policy Optimization Algorithms
PPO 简单稳定，既是机器人 RL 主力，也是 RLHF 的核心算法。
John Schulman, Filip Wolski, Prafulla Dhariwal +2
强化学习、世界模型与机器人算法卡片 →
-
2023Conference on Robot Learning❝4,813
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
RT-2 把视觉语言模型直接变成动作模型，提出 VLA 概念。
Anthony Brohan, Noah Brown, Justice Carbajal +51
强化学习、世界模型与机器人
-
2016arXiv.org❝3,583
Concrete Problems in AI Safety
《AI 安全的具体问题》：奖励黑客、安全探索等问题的经典梳理。
Dario Amodei, Chris Olah, Jacob Steinhardt +3
安全与可解释性
-
2021Nature❝40,064
Highly accurate protein structure prediction with AlphaFold
AlphaFold2 以接近实验精度预测蛋白质结构，AI for Science 的里程碑。
John Jumper, Richard Evans, Alexander Pritzel +31
AI for Science算法卡片 →
-
20092009 IEEE Conference on Computer Vision and Pattern Recognition❝76,917
ImageNet: A large-scale hierarchical image database
ImageNet 大规模图像数据集，深度学习爆发的数据基础。
Jia Deng, Wei Dong, Richard Socher +3
数据集与评测
-
2020International Conference on Learning Representations❝10,132
Measuring Massive Multitask Language Understanding
MMLU 覆盖 57 个学科的知识测评，大模型最常引用的基准之一。
Dan Hendrycks, Collin Burns, Steven Basart +4
数据集与评测
核心 71
-
2013International Conference on Learning Representations❝17,260
Auto-Encoding Variational Bayes
VAE 把概率推断和神经网络结合，是潜变量生成模型的基础。
Diederik P Kingma, Max Welling
深度学习基础算法卡片 →
-
2014
Dropout: A Simple Way to Prevent Neural Networks from Overfitting
Dropout 用随机丢弃神经元抑制过拟合，至今仍是常用正则化。
深度学习基础ⓘ
-
2014Communications of the ACM
Generative Adversarial Networks
GAN 用生成器与判别器博弈来生成数据，开启生成模型新路线。
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza +5
深度学习基础算法卡片 →
-
2014International Conference on Learning Representations❝114,960
Very Deep Convolutional Networks for Large-Scale Image Recognition
VGG 证明“更深的小卷积核网络”更强，成为常用骨干。
Karen Simonyan, Andrew Zisserman
深度学习基础
-
2015International Conference on Machine Learning❝48,068
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
批归一化让深层网络训练更快更稳。
Sergey Ioffe, Christian Szegedy
深度学习基础算法卡片 →
-
2015arXiv.org❝26,906
Distilling the Knowledge in a Neural Network
知识蒸馏：用大模型“软标签”教小模型，今天的模型压缩和蒸馏都源于此。
Geoffrey Hinton, Oriol Vinyals, Jeff Dean
深度学习基础算法卡片 →
-
2016arXiv.org❝13,354
Layer Normalization
LayerNorm 是 Transformer 中标准的归一化层。
Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton
深度学习基础算法卡片 →
-
2018North American Chapter of the Association for Computational Linguistics❝12,330
Deep contextualized word representations
ELMo 证明上下文相关的预训练词表示能普遍提升 NLP 任务。
Matthew E. Peters, Mark Neumann, Mohit Iyyer +4
Transformer 与大模型预训练算法卡片 →
-
2019Journal of machine learning research❝27,824
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
T5 把所有 NLP 任务统一成“文本到文本”，并系统比较了预训练选择。
Colin Raffel, Noam Shazeer, Adam Roberts +6
Transformer 与大模型预训练算法卡片 →
-
2019
Language Models are Unsupervised Multitask Learners
GPT-2 展示了大规模语言模型的零样本多任务能力。
Transformer 与大模型预训练ⓘ
-
2021Neurocomputing❝6,872
RoFormer: Enhanced Transformer with Rotary Position Embedding
RoPE 旋转位置编码，被 LLaMA、Qwen 等主流模型采用。
Jianlin Su, Yu Lu, Shengfeng Pan +3
Transformer 与大模型预训练算法卡片 →
-
2022Journal of machine learning research❝8,497
PaLM: Scaling Language Modeling with Pathways
PaLM 用 Pathways 训练 5400 亿参数模型，展示规模带来的推理进步。
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin +64
Transformer 与大模型预训练
-
2022Advances in Neural Information Processing Systems 35❝3,916
Training Compute-Optimal Large Language Models
Chinchilla 修正了缩放定律：同等算力下数据应与参数同步扩大。
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch +19
Transformer 与大模型预训练算法卡片 →
-
2023❝27,798
GPT-4 Technical Report
GPT-4 技术报告：多模态前沿模型与可预测扩展。
OpenAI, Josh Achiam, Steven Adler +278
Transformer 与大模型预训练
-
2023arXiv.org❝18,643
Llama 2: Open Foundation and Fine-Tuned Chat Models
Llama 2 公开了对话模型及其 RLHF 训练细节。
Hugo Touvron, Louis Martin, Kevin Stone +65
Transformer 与大模型预训练
-
2024
DeepSeek-V3 Technical Report
DeepSeek-V3 技术报告：低成本训练 671B MoE，FP8 训练与多 token 预测。
DeepSeek-AI, Aixin Liu, Bei Feng +197
Transformer 与大模型预训练
-
2024❝19,961
The Llama 3 Herd of Models
Llama 3 系列的完整技术报告，涵盖数据、训练和后训练全流程。
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri +558
Transformer 与大模型预训练
-
2017Neural Information Processing Systems❝6,204
Deep reinforcement learning from human preferences
从人类偏好中学习奖励的深度强化学习，RLHF 的起点。
Paul Christiano, Jan Leike, Tom B. Brown +3
指令微调与对齐算法卡片 →
-
2020Neural Information Processing Systems❝3,579
Learning to summarize from human feedback
用人类反馈训练摘要模型，验证了 RLHF 在语言任务上的效果。
Nisan Stiennon, Long Ouyang, Jeff Wu +6
指令微调与对齐
-
2021International Conference on Learning Representations❝5,525
Finetuned Language Models Are Zero-Shot Learners
FLAN：指令微调让模型零样本完成未见任务。
Jason Wei, Maarten Bosma, Vincent Y. Zhao +6
指令微调与对齐算法卡片 →
-
2022arXiv.org❝3,869
Constitutional AI: Harmlessness from AI Feedback
宪法式 AI：用一组原则让 AI 自我批评和修正，减少人工标注。
Yuntao Bai, Saurav Kadavath, Sandipan Kundu +48
指令微调与对齐算法卡片 →
-
2022arXiv.org❝4,407
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
Anthropic 用 RLHF 训练“有用且无害”的助手，并公开 HH 数据。
Yuntao Bai, Andy Jones, Kamal Ndousse +28
指令微调与对齐
-
2021arXiv.org❝11,505
Training Verifiers to Solve Math Word Problems
GSM8K 数据集与“验证器”方法，数学推理评测的常用基准。
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian +9
推理与推理模型
-
2022Neural Information Processing Systems❝8,280
Large Language Models are Zero-Shot Reasoners
“让我们一步步思考”一句话就能激发零样本推理。
Takeshi Kojima, Shixiang Shane Gu, Machel Reid +2
推理与推理模型
-
2022International Conference on Learning Representations❝8,018
Self-Consistency Improves Chain of Thought Reasoning in Language Models
自洽性：多次采样推理路径再投票，进一步提升准确率。
Xuezhi Wang, Jason Wei, Dale Schuurmans +5
推理与推理模型算法卡片 →
-
2023International Conference on Learning Representations❝5,067
Let's Verify Step by Step
过程监督（逐步打分）比只看结果更能训练可靠的数学推理，并发布 PRM800K。
Hunter Lightman, Vineet Kosaraju, Yura Burda +7
推理与推理模型算法卡片 →
-
2024arXiv.org❝10,174
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
DeepSeekMath 提出 GRPO 强化学习算法，后来成为推理模型训练的主流方法。
Zhihong Shao, Peiyi Wang, Qihao Zhu +8
推理与推理模型算法卡片 →
-
2024arXiv.org❝2,310
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
测试时扩展：在推理阶段多花算力，有时比放大模型更划算。
Charlie Snell, Jaehoon Lee, Kelvin Xu +1
推理与推理模型算法卡片 →
-
2020Neural Information Processing Systems❝19,728
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
RAG：检索增强生成，把外部知识库接到生成模型上。
Patrick Lewis, Ethan Perez, Aleksandra Piktus +9
智能体、工具与代码算法卡片 →
-
2021arXiv.org❝12,247
Evaluating Large Language Models Trained on Code
Codex 与 HumanEval：代码大模型和代码评测的起点。
Mark Chen, Jerry Tworek, Heewoo Jun +55
智能体、工具与代码
-
2023ACM Symposium on User Interface Software and Technology❝6,051
Generative Agents: Interactive Simulacra of Human Behavior
生成式 Agent：25 个 AI 角色在小镇中模拟可信的人类行为。
Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai +3
智能体、工具与代码算法卡片 →
-
2023Neural Information Processing Systems❝6,163
Reflexion: Language Agents with Verbal Reinforcement Learning
Reflexion：Agent 用语言形式的自我反思从失败中改进。
Noah Shinn, Federico Cassano, Edward Berman +3
智能体、工具与代码算法卡片 →
-
2023Neural Information Processing Systems❝6,231
Toolformer: Language Models Can Teach Themselves to Use Tools
Toolformer：模型自学何时调用计算器、搜索等外部工具。
Timo Schick, Jane Dwivedi-Yu, Roberto Dessì +5
智能体、工具与代码算法卡片 →
-
2024Neural Information Processing Systems❝2,213
SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
SWE-agent 提出“Agent-电脑接口”，显著提升编程 Agent 效果。
John Yang, Carlos E. Jimenez, Alexander Wettig +4
智能体、工具与代码算法卡片 →
-
2015IEEE Transactions on Pattern Analysis and Machine Intelligence❝76,541
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Faster R-CNN 用区域建议网络实现端到端目标检测。
Shaoqing Ren, Kaiming He, Ross Girshick +1
视觉与多模态算法卡片 →
-
2015International Conference on Medical Image Computing and Computer-Assisted Intervention❝103,316
U-Net: Convolutional Networks for Biomedical Image Segmentation
U-Net 编码-解码结构，医学分割和扩散模型都在用。
Olaf Ronneberger, Philipp Fischer, Thomas Brox
视觉与多模态算法卡片 →
-
2015Computer Vision and Pattern Recognition❝48,966
You Only Look Once: Unified, Real-Time Object Detection
YOLO 把检测变成一次回归，实现实时检测。
Joseph Redmon, Santosh Divvala, Ross Girshick +1
视觉与多模态算法卡片 →
-
2022Neural Information Processing Systems❝6,816
Flamingo: a Visual Language Model for Few-Shot Learning
Flamingo 把冻结的视觉和语言模型连起来，实现少样本多模态理解。
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc +24
视觉与多模态算法卡片 →
-
2022International Conference on Machine Learning❝9,082
Robust Speech Recognition via Large-Scale Weak Supervision
Whisper 用 68 万小时弱监督数据训练出鲁棒的多语种语音识别。
Alec Radford, Jong Wook Kim, Tao Xu +3
视觉与多模态算法卡片 →
-
2023IEEE International Conference on Computer Vision❝16,271
Segment Anything
SAM“分割一切”，可提示的通用分割基础模型。
Alexander Kirillov, Eric Mintun, Nikhila Ravi +9
视觉与多模态算法卡片 →
-
2020International Conference on Learning Representations❝12,628
Score-Based Generative Modeling through Stochastic Differential Equations
用随机微分方程统一扩散与分数模型。
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma +3
生成模型（图像/视频）算法卡片 →
-
2021International Conference on Machine Learning❝6,809
Zero-Shot Text-to-Image Generation
DALL·E：用 Transformer 做零样本文生图。
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh +5
生成模型（图像/视频）
-
2022arXiv.org❝7,486
Classifier-Free Diffusion Guidance
无分类器引导，几乎所有文生图模型都在用的采样技巧。
Jonathan Ho, Tim Salimans
生成模型（图像/视频）算法卡片 →
-
2022IEEE International Conference on Computer Vision❝7,769
Scalable Diffusion Models with Transformers
DiT 用 Transformer 替代 U-Net 做扩散，是 Sora 等视频模型的架构基础。
William Peebles, Saining Xie
生成模型（图像/视频）算法卡片 →
-
2023IEEE International Conference on Computer Vision❝8,007
Adding Conditional Control to Text-to-Image Diffusion Models
ControlNet 给文生图加入边缘、姿态等精确控制。
Lvmin Zhang, Anyi Rao, Maneesh Agrawala
生成模型（图像/视频）算法卡片 →
-
2024
Video generation models as world simulators
Sora 技术报告：视频生成模型作为“世界模拟器”。
生成模型（图像/视频）ⓘ
-
2019arXiv.org❝10,779
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
DistilBERT：蒸馏出更小更快的 BERT。
Victor Sanh, Lysandre Debut, Julien Chaumond +1
高效训练与推理
-
2022International Conference on Machine Learning❝2,224
Fast Inference from Transformers via Speculative Decoding
推测解码：小模型起草、大模型验证，无损加速生成。
Yaniv Leviathan, Matan Kalman, Yossi Matias
高效训练与推理算法卡片 →
-
2022Neural Information Processing Systems❝5,621
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
FlashAttention 按 IO 优化注意力计算，长上下文训练的关键。
Tri Dao, Daniel Y. Fu, Stefano Ermon +2
高效训练与推理算法卡片 →
-
2022arXiv.org❝2,771
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
GPTQ 训练后量化，把大模型压到 3–4 比特。
Elias Frantar, Saleh Ashkboos, Torsten Hoefler +1
高效训练与推理算法卡片 →
-
2023Neural Information Processing Systems❝6,034
QLoRA: Efficient Finetuning of Quantized LLMs
QLoRA 在 4 比特量化模型上做 LoRA，单卡即可微调大模型。
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman +1
高效训练与推理算法卡片 →
-
2016Nature❝19,421
Mastering the game of Go with deep neural networks and tree search
AlphaGo 结合深度网络与树搜索，首次击败职业围棋选手。
David Silver, Aja Huang, Chris J. Maddison +17
强化学习、世界模型与机器人算法卡片 →
-
2017Nature❝10,775
Mastering the game of Go without human knowledge
AlphaGo Zero 不用人类棋谱，纯自我对弈超越前代。
David Silver, Julian Schrittwieser, Karen Simonyan +14
强化学习、世界模型与机器人
-
2018arXiv.org❝2,241
World Models
World Models：在学到的“梦境”里训练策略。
David Ha, Jürgen Schmidhuber
强化学习、世界模型与机器人算法卡片 →
-
2022Robotics: Science and Systems Conference❝3,074
RT-1: Robotics Transformer for Real-World Control at Scale
RT-1 用 Transformer 从大规模真实数据学机器人控制。
Anthony Brohan, Noah Brown, Justice Carbajal +48
强化学习、世界模型与机器人
-
2023Robotics: Science and Systems Conference❝4,816
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
扩散策略：用扩散模型生成机器人动作，效果稳定。
Cheng Chi, Zhenjia Xu, Siyuan Feng +5
强化学习、世界模型与机器人
-
2023IEEE International Conference on Robotics and Automation❝1,247
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Open X-Embodiment：跨 22 种机器人的大规模开放数据集。
Open X-Embodiment Collaboration, Abby O'Neill, Abdul Rehman +291
强化学习、世界模型与机器人
-
2013International Conference on Learning Representations❝17,138
Intriguing properties of neural networks
发现神经网络对微小扰动极其敏感（对抗样本）。
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever +4
安全与可解释性
-
2014International Conference on Learning Representations❝23,415
Explaining and Harnessing Adversarial Examples
FGSM 解释并高效生成对抗样本，催生对抗训练。
Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy
安全与可解释性算法卡片 →
-
2021Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency❝9,060
On the Dangers of Stochastic Parrots
“随机鹦鹉”：讨论大语言模型的社会风险与成本。
Emily M. Bender, Timnit Gebru, Angelina McMillan-Major +1
安全与可解释性
-
2021Annual Meeting of the Association for Computational Linguistics❝4,141
TruthfulQA: Measuring How Models Mimic Human Falsehoods
TruthfulQA 衡量模型是否会复述人类常见的错误观念。
Stephanie Lin, Jacob Hilton, Owain Evans
安全与可解释性
-
2022arXiv.org❝1,099
Toy Models of Superposition
叠加态玩具模型：解释神经元为何同时编码多个概念。
Nelson Elhage, Tristan Hume, Catherine Olsson +13
安全与可解释性算法卡片 →
-
2023
Towards Monosemanticity: Decomposing Language Models With Dictionary Learning
用稀疏自编码器把神经元分解成可解释特征。
安全与可解释性ⓘ
-
2023arXiv.org❝3,912
Universal and Transferable Adversarial Attacks on Aligned Language Models
自动生成通用对抗后缀，可越狱多种对齐模型。
Andy Zou, Zifan Wang, Nicholas Carlini +3
安全与可解释性算法卡片 →
-
2022arXiv.org❝449
GraphCast: Learning skillful medium-range global weather forecasting
GraphCast：图神经网络做全球中期天气预报，优于传统数值模式。
Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson +15
AI for Science
-
2023Science❝5,900
Evolutionary-scale prediction of atomic-level protein structure with a language model
ESM-2/ESMFold：蛋白质语言模型直接从序列预测结构。
Zeming Lin, Halil Akin, Roshan Rao +12
AI for Science
-
2024Nature❝14,696
Accurate structure prediction of biomolecular interactions with AlphaFold 3
AlphaFold 3 把预测扩展到蛋白质与核酸、小分子等复合物。
Josh Abramson, Jonas Adler, Jack Dunger +45
AI for Science
-
2024Nature❝1,493
Mathematical discoveries from program search with large language models
FunSearch：大模型 + 进化搜索在数学问题上得到新发现。
Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov +9
AI for Science
-
2018BlackboxNLP@EMNLP❝9,120
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
GLUE 多任务语言理解基准，推动了预训练模型竞赛。
Alex Wang, Amanpreet Singh, Julian Michael +3
数据集与评测
-
2021NeurIPS Datasets and Benchmarks❝6,963
Measuring Mathematical Problem Solving With the MATH Dataset
MATH 竞赛数学数据集，推理模型的重要评测。
Dan Hendrycks, Collin Burns, Saurav Kadavath +5
数据集与评测
-
2024International Conference on Machine Learning❝1,645
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
Chatbot Arena：众包两两对战的大模型排名平台。
Wei-Lin Chiang, Lianmin Zheng, Ying Sheng +8
数据集与评测
进阶 44
-
2017International Conference on Learning Representations❝5,967
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
稀疏门控混合专家（MoE）层，现代 MoE 大模型的源头。
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz +4
Transformer 与大模型预训练算法卡片 →
-
2023arXiv.org❝9,684
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Mamba 选择性状态空间模型，线性复杂度的 Transformer 替代方案。
Albert Gu, Tri Dao
Transformer 与大模型预训练算法卡片 →
-
2023arXiv.org❝4,047
Mistral 7B
Mistral 7B 用滑动窗口注意力和 GQA 做出高效小模型。
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch +15
Transformer 与大模型预训练
-
2024arXiv.org❝1,543
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
DeepSeek-V2 提出 MLA 注意力和 DeepSeekMoE，大幅降低推理成本。
DeepSeek-AI, Aixin Liu, Bei Feng +154
Transformer 与大模型预训练算法卡片 →
-
2024arXiv.org❝2,219
Mixtral of Experts
Mixtral 8x7B 把稀疏 MoE 带入主流开源模型。
Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux +23
Transformer 与大模型预训练算法卡片 →
-
2024arXiv.org❝5,530
Qwen2.5 Technical Report
Qwen2.5 技术报告，代表性的中文开源模型家族。
Qwen, :, An Yang +41
Transformer 与大模型预训练
-
2025arXiv.org❝484
GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
GLM-4.5：智谱面向 Agent、推理与代码的开源 MoE 模型技术报告。
GLM-4. 5 Team, :, Aohan Zeng +169
Transformer 与大模型预训练
-
2025❝427
Kimi K2: Open Agentic Intelligence
Kimi K2：万亿参数开源 MoE，面向 Agent 能力并提出 MuonClip 优化器。
Kimi Team, Yifan Bai, Yiping Bao +197
Transformer 与大模型预训练
-
2025❝9,355
Qwen3 Technical Report
Qwen3 技术报告：思考/非思考模式融合于一个模型。
An Yang, Anfeng Li, Baosong Yang +57
Transformer 与大模型预训练
-
2022International Conference on Machine Learning❝1,402
Scaling Laws for Reward Model Overoptimization
研究奖励模型被过度优化（奖励黑客）的缩放规律。
Leo Gao, John Schulman, Jacob Hilton
指令微调与对齐
-
2023International Conference on Machine Learning❝573
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
弱到强泛化：用弱模型监督强模型，探索超人类模型的对齐问题。
Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner +9
指令微调与对齐算法卡片 →
-
2023Neural Information Processing Systems❝5,166
Tree of Thoughts: Deliberate Problem Solving with Large Language Models
思维树：把推理做成树搜索，可回溯和比较多条思路。
Shunyu Yao, Dian Yu, Jeffrey Zhao +4
推理与推理模型算法卡片 →
-
2024
OpenAI o1 System Card
OpenAI o1 系统卡，首个大规模“推理模型”的能力与安全评估。
OpenAI, :, Aaron Jaech +262
推理与推理模型
-
2024Nature❝917
Solving olympiad geometry without human demonstrations
AlphaGeometry 不靠人类示范，以奥赛级水平解几何题。
Trieu H. Trinh, Yuhuai Wu, Quoc V. Le +2
推理与推理模型算法卡片 →
-
2025Neural Information Processing Systems❝2,988
DAPO: An Open-Source LLM Reinforcement Learning System at Scale
DAPO：开源的大规模推理强化学习系统与训练技巧。
Qiying Yu, Zheng Zhang, Ruofei Zhu +32
推理与推理模型算法卡片 →
-
2025arXiv.org❝1,117
Kimi k1.5: Scaling Reinforcement Learning with LLMs
Kimi k1.5：长上下文强化学习扩展推理能力的技术报告。
Kimi Team, Angang Du, Bofei Gao +93
推理与推理模型
-
2025Conference on Empirical Methods in Natural Language Processing❝1,561
s1: Simple test-time scaling
s1：只用 1000 条数据加“预算强制”就能实现测试时扩展。
Niklas Muennighoff, Zitong Yang, Weijia Shi +7
推理与推理模型算法卡片 →
-
2022Science❝2,634
Competition-Level Code Generation with AlphaCode
AlphaCode 在编程竞赛中达到中等人类选手水平。
Yujia Li, David Choi, Junyoung Chung +23
智能体、工具与代码
-
2023International Conference on Learning Representations❝2,322
WebArena: A Realistic Web Environment for Building Autonomous Agents
WebArena：可复现的真实网站环境，用于评测网页 Agent。
Shuyan Zhou, Frank F. Xu, Hao Zhu +9
智能体、工具与代码
-
2024Neural Information Processing Systems❝1,369
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
OSWorld：真实操作系统中的电脑操作 Agent 基准。
Tianbao Xie, Danyang Zhang, Jixuan Chen +14
智能体、工具与代码
-
2020International Conference on Machine Learning❝26,751
A Simple Framework for Contrastive Learning of Visual Representations
SimCLR 对比学习，无标注也能学到好的视觉表示。
Ting Chen, Simon Kornblith, Mohammad Norouzi +1
视觉与多模态算法卡片 →
-
2020European Conference on Computer Vision
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
NeRF 用神经网络表示 3D 场景，实现高质量新视角合成。
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik +3
视觉与多模态算法卡片 →
-
2023ACM Transactions on Graphics❝10,958
3D Gaussian Splatting for Real-Time Radiance Field Rendering
3D 高斯泼溅实现实时高质量的新视角渲染。
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler +1
视觉与多模态
-
2023International Conference on Machine Learning❝9,548
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
BLIP-2 用轻量 Q-Former 连接图像编码器和大模型。
Junnan Li, Dongxu Li, Silvio Savarese +1
视觉与多模态
-
2023Trans. Mach. Learn. Res.❝11,394
DINOv2: Learning Robust Visual Features without Supervision
DINOv2 自监督学到通用视觉特征，不需微调就可用。
Maxime Oquab, Timothée Darcet, Théo Moutakanni +23
视觉与多模态算法卡片 →
-
2024arXiv.org❝5,261
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
Qwen2-VL 支持任意分辨率的视觉语言模型。
Peng Wang, Shuai Bai, Sinan Tan +16
视觉与多模态
-
2022International Conference on Learning Representations❝7,112
Flow Matching for Generative Modeling
流匹配：比扩散更简洁的生成训练方法，被新一代图像模型采用。
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu +2
生成模型（图像/视频）算法卡片 →
-
2023International Conference on Machine Learning❝2,351
Consistency Models
一致性模型：一步或几步就能生成高质量图像。
Yang Song, Prafulla Dhariwal, Mark Chen +1
生成模型（图像/视频）算法卡片 →
-
2024International Conference on Machine Learning❝5,016
Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
Stable Diffusion 3：整流流 + 多模态 DiT 的规模化。
Patrick Esser, Sumith Kulal, Andreas Blattmann +14
生成模型（图像/视频）
-
2019arXiv.org❝3,294
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Megatron-LM 张量并行，训练超大模型的基础设施。
Mohammad Shoeybi, Mostofa Patwary, Raul Puri +3
高效训练与推理算法卡片 →
-
2019International Conference for High Performance Computing, Networking, Storage and Analysis❝2,160
ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
ZeRO 切分优化器状态，显著降低大模型训练显存。
Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase +1
高效训练与推理算法卡片 →
-
2023Conference on Empirical Methods in Natural Language Processing❝1,925
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
GQA 分组查询注意力，兼顾 MQA 的速度和多头的质量。
Joshua Ainslie, James Lee-Thorp, Michiel de Jong +3
高效训练与推理算法卡片 →
-
2023arXiv.org❝1,520
Mastering Diverse Domains through World Models
DreamerV3 用世界模型在大量领域用同一套超参数取得好成绩。
Danijar Hafner, Jurgis Pasukonis, Jimmy Ba +1
强化学习、世界模型与机器人
-
2024arXiv.org❝3,247
$π_0$: A Vision-Language-Action Flow Model for General Robot Control
π0：基于流匹配的通用机器人策略模型。
Kevin Black, Noah Brown, Danny Driess +21
强化学习、世界模型与机器人
-
2024Conference on Robot Learning❝4,041
OpenVLA: An Open-Source Vision-Language-Action Model
OpenVLA：开源 7B 视觉-语言-动作模型。
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti +15
强化学习、世界模型与机器人
-
2024arXiv.org❝377
Alignment faking in large language models
对齐伪装：模型在训练中表面顺从、实际保留原有偏好。
Ryan Greenblatt, Carson Denison, Benjamin Wright +17
安全与可解释性
-
2024
Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
把可解释特征提取扩展到生产级大模型 Claude 3 Sonnet。
安全与可解释性ⓘ
-
2024arXiv.org❝631
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
“潜伏特工”：后门行为可以在安全训练后依然存在。
Evan Hubinger, Carson Denison, Jesse Mu +36
安全与可解释性算法卡片 →
-
2022Nature❝773
Discovering faster matrix multiplication algorithms with reinforcement learning
AlphaTensor 用强化学习发现更快的矩阵乘法算法。
Alhussein Fawzi, Matej Balog, Aja Huang +10
AI for Science
-
2023Nature❝1,839
Scaling deep learning for materials discovery
GNoME 用深度学习发现数十万种稳定晶体材料。
Amil Merchant, Simon Batzner, Samuel S. Schoenholz +3
AI for Science
-
2024arXiv.org❝1,291
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
AI Scientist：全自动完成选题、实验和写论文的尝试。
Chris Lu, Cong Lu, Robert Tjarko Lange +3
AI for Science
-
2025arXiv.org❝1,067
AlphaEvolve: A coding agent for scientific and algorithmic discovery
AlphaEvolve：编程 Agent 进化算法，用于科学和算法发现。
Alexander Novikov, Ngân Vũ, Marvin Eisenberger +15
AI for Science
-
2023arXiv.org❝3,721
GPQA: A Graduate-Level Google-Proof Q&A Benchmark
GPQA：“谷歌不到答案”的研究生级科学问答。
David Rein, Betty Li Hou, Asa Cooper Stickland +5
数据集与评测
-
2025Nature❝422
Humanity's Last Exam
Humanity's Last Exam：面向前沿模型的高难度多学科考试。
Long Phan, Alice Gatti, Ziwen Han +1178
数据集与评测