Skip to content
English

Switch theme

Discover▾

Not sure where to start? This week’s hottest software, models and stacks in three minutes.

AI Apps▾

AI software people use: from assistants like ChatGPT to tools for coding, video and workflows, picked by purpose.

For Agents▾

Software for AI: tools that let your agent (Claude Code, Codex, Cursor…) browse, read docs, run code and remember.

Models▾

Choose models, estimate cost, self-host: prices, rankings, benchmarks, free tiers and compute.

Solutions▾

A goal or an error message? Ready-made stacks, steps and fixes by scenario.

Learn▾

A systematic course from zero to the frontier, with an algorithm library, must-read papers and tutorials.

Frontier▾

Daily AI news, weekly highlights, the latest papers and how AI is changing every discipline.

Knowledge · %

MMLU

About 14,000 multiple-choice questions over 57 subjects, from elementary maths to law and medicine.

Saturated Contamination risk: High 198 results · 158 models Updated Oct 22, 2026 Newest result: 2024-12-26 · stale Official leaderboard ↗

What it measures

Broad academic knowledge (usually 5-shot exact match). Data compiled by Epoch AI from papers, model reports and HELM.

Limitations

Saturated at the top; studies found a share of mislabelled questions; prompt format changes scores by several points.

Contamination risk: High

Public since 2020 and widely copied on the web.

Leaderboard · Subset / setting: compiled

96 models

# Model Provider Score Rank Setting Source type Date
1 GPT-4o (Nov 2024) OpenAI 88.1% 1 Exact match; 5-shot; effort: Nov 2024 Paper / tech report via Epoch AI Nov 20, 2024
2 DeepSeek-V3OW DeepSeek 87.1% 2 Exact match; 5-shot Paper / tech report via Epoch AI Dec 26, 2024
3 GPT-4 (Mar 2023) OpenAI 86.4% 3 Exact match; 5-shot; effort: Mar 2023 Paper / tech report via Epoch AI Mar 14, 2023
4 Llama 3.3 70BOW Meta 86.3% 4 Exact match; 5-shot Paper / tech report via Epoch AI Dec 6, 2024
5 Gemini 1.5 Pro (May 2024) Google 85.9% 5 Exact match; 5-shot Vendor-reported via Epoch AI May 14, 2024
6 Qwen2.5-72BOW Alibaba (Qwen) 85.0% 6 Exact match; 5-shot Paper / tech report via Epoch AI Sep 19, 2024
7 Phi-4OW Microsoft Research 84.8% 7 Exact match; 5-shot Paper / tech report via Epoch AI Dec 12, 2024
8 Llama 3.1-405BOW Meta 84.4% 8 Exact match; 5-shot Paper / tech report via Epoch AI Jul 23, 2024
9 GPT-4 OpenAI 83.9% 9 Exact match; 5-shot Paper / tech report via Epoch AI —
10 Gemini 1.5 Pro (Feb 2024) Google 81.9% 10 Exact match; 5-shot Vendor-reported via Epoch AI Feb 15, 2024
11 GPT-4o mini OpenAI 81.8% 11 Exact match; 5-shot Paper / tech report via Epoch AI Jul 18, 2024
12 Qwen2.5-14BOW Alibaba (Qwen) 79.9% 12 Exact match; 5-shot Paper / tech report via Epoch AI Sep 19, 2024
13 Claude 2 Anthropic 78.5% 13 Exact match; 5-shot Vendor-reported via Epoch AI Jul 11, 2023
14 DeepSeek-V2 (MoE-236B, May 2024)OW DeepSeek 78.4% 14 Exact match; 5-shot Paper / tech report via Epoch AI May 7, 2024
15 phi-3-medium 14BOW Microsoft 77.9% 15 Exact match; 5-shot Paper / tech report via Epoch AI Apr 23, 2024
16 Claude 1.3 Anthropic 77.0% 16 Exact match; 5-shot Vendor-reported via Epoch AI Apr 18, 2023
17 Yi-34BOW 01.AI 76.3% 17 Exact match; 5-shot Paper / tech report via Epoch AI Nov 2, 2023
18 phi-3-small 7.4BOW Microsoft 75.7% 18 Exact match; 5-shot Paper / tech report via Epoch AI Apr 23, 2024
19 Qwen2.5-Coder-14BOW Alibaba (Qwen) 75.2% 19 Exact match Paper / tech report via Epoch AI Sep 18, 2024
20 Claude Instant Anthropic 73.4% 20 Exact match; 5-shot Vendor-reported via Epoch AI —
21 Inflection-1 Inflection AI 72.7% 21 Exact match; 5-shot Vendor-reported via Epoch AI Jun 22, 2023
22 GPT-3.5 Turbo (Nov 2023) OpenAI 71.4% 22 Exact match; 5-shot; effort: Nov 2023 Paper / tech report via Epoch AI Nov 6, 2023
23 Falcon-180BOW Technology Innovation Institute 70.6% 23 Exact match; 5-shot Paper / tech report via Epoch AI Sep 6, 2023
24 Mixtral 8x7BOW Mistral AI 70.6% 23 Exact match; 5-shot Paper / tech report via Epoch AI Dec 11, 2023
25 davinci-002 OpenAI 70.0% 25 Exact match; 5-shot Paper / tech report via Epoch AI Mar 15, 2022
26 PaLM (540B) Google Research 69.3% 26 Exact match; 5-shot Paper / tech report via Epoch AI Apr 4, 2022
27 Llama 2-70BOW Meta 68.9% 27 Exact match; 5-shot Paper / tech report via Epoch AI Jul 18, 2023
28 phi-3-mini 3.8BOW Microsoft 68.8% 28 Exact match; 5-shot Paper / tech report via Epoch AI Apr 23, 2024
29 Stable Beluga 2OW Stability AI 68.6% 29 Exact match; 5-shot Paper / tech report via Epoch AI Jul 20, 2023
30 GPT-3.5 Turbo OpenAI 68.5% 30 Exact match; 5-shot Paper / tech report via Epoch AI —
31 Yi-9BOW 01.AI 68.4% 31 Exact match; 5-shot Paper / tech report via Epoch AI Mar 1, 2024
32 Qwen2.5-Coder (7B)OW Alibaba (Qwen) 68.0% 32 Exact match Paper / tech report via Epoch AI Sep 18, 2024
33 Chinchilla Google 67.5% 34 Exact match; 5-shot Paper / tech report via Epoch AI Mar 29, 2022
34 InternLM (104B) Shanghai AI Lab (InternLM) 67.2% 35 Exact match; 5-shot Vendor-reported via Epoch AI —
35 AquilaChat2 34B — 66.7% 36 Exact match; 5-shot Paper / tech report via Epoch AI —
36 Llama 3-8BOW Meta 66.5% 37 Exact match; 5-shot Paper / tech report via Epoch AI Apr 18, 2024
37 Qwen-14BOW Alibaba (Qwen) 66.3% 38 Exact match; 5-shot Paper / tech report via Epoch AI Sep 28, 2023
38 Gemma 7BOW Google 64.3% 40 Exact match; 5-shot Paper / tech report via Epoch AI Feb 21, 2024
39 StarCoder 2 15BOW Hugging Face 64.1% 41 Exact match Paper / tech report via Epoch AI Feb 20, 2024
40 LLaMA-65BOW Meta 63.4% 43 Exact match; 5-shot Paper / tech report via Epoch AI Feb 24, 2023
41 Yi 6BOW 01.AI 63.2% 44 Exact match; 5-shot Paper / tech report via Epoch AI Nov 2, 2023
42 Llama 2-34B Meta 62.6% 45 Exact match; 5-shot Paper / tech report via Epoch AI Jul 18, 2023
43 Mistral 7B v0.2OW Mistral AI 62.5% 46 Exact match; 5-shot Paper / tech report via Epoch AI Dec 11, 2023
44 Mistral 7B v0.1OW Mistral AI 62.5% 46 Exact match; 5-shot Paper / tech report via Epoch AI Sep 27, 2023
45 Qwen2.5-Coder-32BOW Alibaba (Qwen) 61.2% 48 Exact match Paper / tech report via Epoch AI Sep 18, 2024
46 DeepSeek-Coder-V2-Lite-Base DeepSeek 60.5% 49 Exact match Paper / tech report via Epoch AI Jun 13, 2024
47 Gopher (280B) Google 60.0% 50 Exact match; 5-shot Paper / tech report via Epoch AI Dec 8, 2021
48 Baichuan2-13BOW Baichuan 59.2% 52 Exact match; 5-shot Paper / tech report via Epoch AI Sep 6, 2023
49 Nemotron-4 15B NVIDIA 58.7% 53 Exact match; 5-shot Paper / tech report via Epoch AI Feb 26, 2024
50 Falcon 2 11BOW Technology Innovation Institute 58.4% 54 Exact match; 5-shot Paper / tech report via Epoch AI May 9, 2024
51 LLaMA-33BOW Meta 57.8% 56 Exact match; 5-shot Paper / tech report via Epoch AI Feb 24, 2023
52 internlm-chat-20b Shanghai AI Lab (InternLM) 57.4% 57 Exact match; 5-shot Paper / tech report via Epoch AI Sep 17, 2023
53 Falcon-40BOW Technology Innovation Institute 56.9% 58 Exact match; 5-shot Paper / tech report via Epoch AI Mar 15, 2023
54 Phi-2OW Microsoft 56.3% 59 Exact match; 5-shot Paper / tech report via Epoch AI Dec 12, 2023
55 XVERSE-13B — 55.2% 61 Exact match; 5-shot Paper / tech report via Epoch AI —
56 Baichuan2-13B-Chat Baichuan 55.1% 62 Exact match; 0-shot Paper / tech report via Epoch AI Sep 6, 2023
57 Llama 2-13BOW Meta 54.8% 63 Exact match; 5-shot Paper / tech report via Epoch AI Jul 18, 2023
58 Baichuan 2-7BOW Baichuan 54.2% 64 Exact match; 5-shot Paper / tech report via Epoch AI Sep 20, 2023
59 PaLM 62B Google 53.7% 65 Exact match; 5-shot Paper / tech report via Epoch AI Apr 4, 2022
60 Qwen2.5-Coder (1.5B)OW Alibaba (Qwen) 53.6% 66 Exact match Paper / tech report via Epoch AI Sep 18, 2024
61 Vicuna-13BOW LMSYS 52.0% 67 Exact match; 5-shot Paper / tech report via Epoch AI —
62 Baichuan 1-13BOW Baichuan 51.6% 68 Exact match; 5-shot Paper / tech report via Epoch AI Jul 11, 2023
63 internlm-7b Shanghai AI Lab (InternLM) 51.0% 69 Exact match; 5-shot Paper / tech report via Epoch AI Jul 5, 2023
64 INTELLECT-1 Prime Intellect 49.9% 72 Exact match; 5-shot Paper / tech report via Epoch AI Nov 29, 2024
65 chatglm2-6bOW Zhipu AI (Z.ai) 47.9% 73 Exact match; 5-shot Paper / tech report via Epoch AI Jun 24, 2023
66 MPT-30BOW MosaicML 47.9% 73 Exact match; 5-shot Paper / tech report via Epoch AI Jun 22, 2023
67 ChatGLM 2-6B (base)* Zhipu AI (Z.ai) 47.9% 75 Exact match; 5-shot Paper / tech report via Epoch AI —
68 LLaMA-13BOW Meta 46.9% 76 Exact match; 5-shot Paper / tech report via Epoch AI Feb 24, 2023
69 Llama 2-7BOW Meta 45.3% 77 Exact match; 5-shot Paper / tech report via Epoch AI Jul 18, 2023
70 Qwen-7BOW Alibaba (Qwen) 45.0% 79 Exact match; 5-shot Paper / tech report via Epoch AI Sep 28, 2023
71 Chinese-Alpaca-Plus-13B Community (Chinese-LLaMA) 43.9% 80 Exact match; 5-shot Paper / tech report via Epoch AI —
72 InstructGPT 175B OpenAI 43.9% 80 Exact match; 5-shot Paper / tech report via Epoch AI Jan 27, 2022
73 Baichuan1-7BOW Baichuan 42.3% 82 Exact match; 5-shot Paper / tech report via Epoch AI Jun 1, 2023
74 Gemma 2BOW Google 42.3% 82 Exact match; 5-shot Paper / tech report via Epoch AI Feb 21, 2024
75 Qwen2.5-Coder-0.5B Alibaba (Qwen) 42.0% 84 Exact match Paper / tech report via Epoch AI Sep 18, 2024
76 CodeQwen1.5-7BOW Alibaba (Qwen) 40.5% 85 Exact match Paper / tech report via Epoch AI Apr 15, 2024
77 DeepSeek Coder 33BOW DeepSeek 39.4% 86 Exact match Paper / tech report via Epoch AI Nov 2, 2023
78 StarCoder 2 7BOW Hugging Face 38.8% 87 Exact match Paper / tech report via Epoch AI Feb 20, 2024
79 Phi-1.5OW Microsoft 37.6% 88 Exact match; 2-shot Paper / tech report via Epoch AI Sep 11, 2023
80 StarCoder 2 3BOW Hugging Face 36.6% 89 Exact match Paper / tech report via Epoch AI Feb 22, 2024
81 DeepSeek Coder 6.7BOW DeepSeek 36.4% 90 Exact match Paper / tech report via Epoch AI Nov 2, 2023
82 XGen-7BOW Salesforce 36.3% 91 Exact match; 5-shot Paper / tech report via Epoch AI Jun 27, 2023
83 LLaMA-7BOW Meta 35.2% 92 Exact match; 2-shot Paper / tech report via Epoch AI Feb 24, 2023
84 Falcon-7BOW Technology Innovation Institute 35.0% 94 Exact match; 5-shot Paper / tech report via Epoch AI Apr 24, 2023
85 GPT-NeoX 20BOW EleutherAI 33.6% 95 Exact match; 5-shot Paper / tech report via Epoch AI —
86 MPT-7BOW MosaicML 30.8% 98 Exact match; 5-shot Paper / tech report via Epoch AI May 5, 2023
87 open_llama_7b — 29.9% 99 Exact match; 5-shot Paper / tech report via Epoch AI Jun 7, 2023
88 Qwen-1_8B Alibaba (Qwen) 28.2% 101 Exact match; 5-shot Paper / tech report via Epoch AI Nov 30, 2023
89 RedPajama-INCITE-7B-Base — 26.3% 105 Exact match; 5-shot Paper / tech report via Epoch AI May 4, 2023
90 Cerebras-GPT-13BOW Cerebras Systems 26.2% 106 Exact match; 5-shot Paper / tech report via Epoch AI Mar 20, 2023
91 Dolly 2.0-12bOW Databricks 26.2% 106 Exact match; 5-shot Paper / tech report via Epoch AI Apr 11, 2023
92 Falcon-rw-1.3B TII 25.9% 108 Exact match; 2-shot Paper / tech report via Epoch AI —
93 DeepSeek Coder 1.3BOW DeepSeek 25.8% 109 Exact match Paper / tech report via Epoch AI Nov 2, 2023
94 GPT-J-6BOW EleutherAI 25.7% 111 Exact match; 0-shot Paper / tech report via Epoch AI Aug 5, 2021
95 PaLM 8B Google 25.4% 112 Exact match; 5-shot Paper / tech report via Epoch AI —
96 opt-13b — 25.1% 114 Exact match; 5-shot Paper / tech report via Epoch AI May 11, 2022

Leaderboard · Subset / setting: leaderboard

79 models

# Model Provider Score Rank Setting Source type Date
1 Claude 3.5 Sonnet (October 2024) Anthropic 87.3% 1 Exact match; 5-shot; effort: Oct 2024 Official leaderboard via Epoch AI Oct 22, 2024
2 DeepSeek-V3OW DeepSeek 87.2% 2 Exact match; 5-shot Official leaderboard via Epoch AI Dec 26, 2024
3 Gemini 1.5 Pro (Sept 2024) Google 86.9% 3 Exact match; 5-shot Official leaderboard via Epoch AI Sep 24, 2024
4 Claude 3.5 Sonnet Anthropic 86.5% 4 Exact match; 5-shot; effort: Jun 2024 Official leaderboard via Epoch AI Jun 20, 2024
5 Claude 3 Opus Anthropic 84.6% 5 Exact match; 5-shot Official leaderboard via Epoch AI Feb 29, 2024
6 Llama 3.1-405BOW Meta 84.5% 6 Exact match; 5-shot Official leaderboard via Epoch AI Jul 23, 2024
7 GPT-4o (Aug 2024) OpenAI 84.3% 7 Exact match; 5-shot; effort: Aug 2024 Official leaderboard via Epoch AI Aug 6, 2024
8 GPT-4o (May 2024) OpenAI 84.2% 8 Exact match; 5-shot; effort: May 2024 Official leaderboard via Epoch AI May 13, 2024
9 Qwen2.5-72BOW Alibaba (Qwen) 83.4% 9 Exact match; 5-shot Official leaderboard via Epoch AI Sep 19, 2024
10 Gemini 1.5 Pro (Feb 2024) Google 82.7% 10 Exact match; 5-shot Official leaderboard via Epoch AI Feb 15, 2024
11 GPT-4 (Jun 2023) OpenAI 82.4% 11 Exact match; 5-shot; effort: Jun 2023 Official leaderboard via Epoch AI Jun 13, 2023
12 Qwen2-72BOW Alibaba (Qwen) 82.4% 11 Exact match; 5-shot Official leaderboard via Epoch AI Jun 7, 2024
13 Amazon Nova Pro Amazon 82.0% 13 Exact match; 5-shot Official leaderboard via Epoch AI Dec 3, 2024
14 GPT-4 Turbo (Apr 2024) OpenAI 81.3% 14 Exact match; 5-shot Official leaderboard via Epoch AI Apr 9, 2024
15 Gemini 1.5 Pro (0409 preview) Google 81.0% 15 Exact match; 5-shot Official leaderboard via Epoch AI —
16 Llama 3.2 90BOW Meta 80.3% 16 Exact match; 5-shot Official leaderboard via Epoch AI Sep 24, 2024
17 Llama 3.1-70BOW Meta 80.1% 17 Exact match; 5-shot Official leaderboard via Epoch AI Jul 23, 2024
18 Mistral Large 2 (Jul 2024)OW Mistral AI 80.0% 18 Exact match; 5-shot Official leaderboard via Epoch AI Jul 24, 2024
19 Gemini 2.0 Flash (Dec 2024) Google 79.7% 19 Exact match; 5-shot Official leaderboard via Epoch AI Dec 11, 2024
20 GPT-4 Turbo (Nov 2023) OpenAI 79.6% 20 Exact match; 5-shot Official leaderboard via Epoch AI Nov 6, 2023
21 Llama 3-70BOW Meta 79.3% 21 Exact match; 5-shot Official leaderboard via Epoch AI Apr 18, 2024
22 Yi-Large 01.AI 79.3% 21 Exact match; 5-shot Official leaderboard via Epoch AI May 13, 2024
23 Llama 3.3 70BOW Meta 79.1% 23 Exact match; 5-shot Official leaderboard via Epoch AI Dec 6, 2024
24 PaLM-2 (Unicorn) Google 78.6% 24 Exact match; 5-shot Official leaderboard via Epoch AI —
25 Palmyra X V3 (72B) Google 78.6% 24 Exact match; 5-shot Official leaderboard via Epoch AI —
26 Jamba 1.5 LargeOW AI21 Labs 78.2% 26 Exact match; 5-shot Official leaderboard via Epoch AI —
27 Gemini 1.5 Flash (May 2024) Google 77.9% 27 Exact match; 5-shot; effort: May 2024 Official leaderboard via Epoch AI May 23, 2024
28 Mixtral 8x22BOW Mistral AI 77.8% 28 Exact match; 5-shot Official leaderboard via Epoch AI Apr 17, 2024
29 Solar Pro — 77.6% 30 Exact match; 5-shot Official leaderboard via Epoch AI —
30 phi-3-medium 14BOW Microsoft 77.5% 31 Exact match; 5-shot Official leaderboard via Epoch AI Apr 23, 2024
31 Qwen1.5-72BOW Alibaba (Qwen) 77.4% 32 Exact match; 5-shot“Qwen1.5 (72B)” Official leaderboard via Epoch AI —
32 Amazon Nova Lite Amazon 77.0% 33 Exact match; 5-shot Official leaderboard via Epoch AI Dec 3, 2024
33 Qwen1.5 Chat (110B) Alibaba (Qwen) 76.8% 34 Exact match; 5-shot Official leaderboard via Epoch AI —
34 GPT-4o mini OpenAI 76.7% 35 Exact match; 5-shot Official leaderboard via Epoch AI Jul 18, 2024
35 Yi-34BOW 01.AI 76.2% 36 Exact match; 5-shot Official leaderboard via Epoch AI Nov 2, 2023
36 Claude 3 Sonnet Anthropic 75.9% 37 Exact match; 5-shot Official leaderboard via Epoch AI Feb 29, 2024
37 Gemma 2 27BOW Google 75.7% 38 Exact match; 5-shot Official leaderboard via Epoch AI Jun 24, 2024
38 Phi-3 (7B) Microsoft 75.7% 38 Exact match; 5-shot Official leaderboard via Epoch AI —
39 Qwen1.5-32BOW Alibaba (Qwen) 74.4% 40 Exact match; 5-shot Official leaderboard via Epoch AI Feb 4, 2024
40 Claude 3.5 Haiku Anthropic 74.3% 41 Exact match; 5-shot; effort: Oct 2024 Official leaderboard via Epoch AI Oct 22, 2024
41 DBRXOW Databricks 74.1% 42 Exact match; 5-shot“DBRX Instruct” Official leaderboard via Epoch AI —
42 Gemini 1.5 Flash (Sep 2024) Google 73.9% 43 Exact match; 5-shot Official leaderboard via Epoch AI Sep 24, 2024
43 Palmyra-X-004 Writer 73.9% 43 Exact match; 5-shot Official leaderboard via Epoch AI —
44 Claude 3 Haiku Anthropic 73.8% 45 Exact match; 5-shot Official leaderboard via Epoch AI Mar 7, 2024
45 Claude 2.1 Anthropic 73.5% 46 Exact match; 5-shot Official leaderboard via Epoch AI Nov 21, 2023
46 Qwen2.5-7BOW Alibaba (Qwen) 72.9% 47 Exact match; 5-shot Official leaderboard via Epoch AI Sep 19, 2024
47 DeepSeek LLM Chat (67B) DeepSeek 72.5% 48 Exact match; 5-shot Official leaderboard via Epoch AI —
48 Gemma 2 9BOW Google 72.1% 49 Exact match; 5-shot Official leaderboard via Epoch AI Jun 24, 2024
49 Mixtral (8x7B 32K seqlen) Mistral AI 71.7% 50 Exact match; 5-shot Official leaderboard via Epoch AI —
50 Amazon Nova Micro Amazon 70.8% 51 Exact match; 5-shot Official leaderboard via Epoch AI Dec 3, 2024
51 Gemini 1.0 Pro Google 70.0% 52 Exact match; 5-shot Official leaderboard via Epoch AI Dec 13, 2023
52 Jamba 1.5 MiniOW AI21 Labs 69.9% 53 Exact match; 5-shot Official leaderboard via Epoch AI —
53 Llama 2-70BOW Meta 69.5% 54 Exact match; 5-shot Official leaderboard via Epoch AI Jul 18, 2023
54 Command R+OW Cohere 69.4% 55 Exact match; 5-shot Official leaderboard via Epoch AI Aug 30, 2024
55 PaLM-2 (Bison) Google 69.2% 56 Exact match; 5-shot Official leaderboard via Epoch AI —
56 GPT-3.5 Turbo (Jun 2023) OpenAI 68.9% 57 Exact match; 5-shot; effort: June 2023 Official leaderboard via Epoch AI Jun 13, 2023
57 Claude Instant Anthropic 68.8% 58 Exact match; 5-shot Official leaderboard via Epoch AI Aug 9, 2023
58 Llama 3-8BOW Meta 68.8% 58 Exact match; 5-shot Official leaderboard via Epoch AI Apr 18, 2024
59 Mistral Large Mistral AI 68.8% 58 Exact match; 5-shot Official leaderboard via Epoch AI Feb 26, 2024
60 mistral-small-2402 Mistral AI 68.7% 61 Exact match; 5-shot Official leaderboard via Epoch AI Feb 26, 2024
61 Qwen1.5-14BOW Alibaba (Qwen) 68.6% 62 Exact match; 5-shot Official leaderboard via Epoch AI Feb 4, 2024
62 Arctic Instruct Snowflake 67.7% 63 Exact match; 5-shot Official leaderboard via Epoch AI —
63 GPT-3.5 Turbo (Jan 2024) OpenAI 67.3% 64 Exact match; 5-shot; effort: Jan 2024 Official leaderboard via Epoch AI Jan 25, 2024
64 Gemma 7BOW Google 66.1% 65 Exact match; 5-shot Official leaderboard via Epoch AI Feb 21, 2024
65 Jamba Instruct AI21 Labs 65.9% 66 Exact match; 5-shot Official leaderboard via Epoch AI —
66 Mistral NeMo (2402) Mistral AI 65.3% 67 Exact match; 5-shot Official leaderboard via Epoch AI —
67 c4ai-command-r-08-2024 Cohere 65.2% 68 Exact match; 5-shot Official leaderboard via Epoch AI Aug 30, 2024
68 Yi 6BOW 01.AI 64.0% 69 Exact match; 5-shot Official leaderboard via Epoch AI Nov 2, 2023
69 Qwen1.5-7BOW Alibaba (Qwen) 62.6% 70 Exact match; 5-shot Official leaderboard via Epoch AI Feb 4, 2024
70 Palmyra X V2 (33B) Google 62.1% 71 Exact match; 5-shot Official leaderboard via Epoch AI —
71 Mistral 7B v0.3OW Mistral AI 59.9% 72 Exact match; 5-shot Official leaderboard via Epoch AI May 27, 2024
72 Phi-2OW Microsoft 58.4% 73 Exact match; 5-shot Official leaderboard via Epoch AI Dec 12, 2023
73 Mistral 7B v0.1OW Mistral AI 56.6% 74 Exact match; 5-shot Official leaderboard via Epoch AI Sep 27, 2023
74 Llama 3.2 11BOW Meta 56.5% 75 Exact match; 5-shot Official leaderboard via Epoch AI Sep 24, 2024
75 Llama 3.1-8BOW Meta 56.1% 76 Exact match; 5-shot Official leaderboard via Epoch AI Jul 23, 2024
76 Llama 2-13BOW Meta 55.4% 77 Exact match; 5-shot Official leaderboard via Epoch AI Jul 18, 2023
77 OLMo 1.7 (7B) Ai2 53.8% 78 Exact match; 5-shot Official leaderboard via Epoch AI —
78 Llama 2-7BOW Meta 45.8% 79 Exact match; 5-shot Official leaderboard via Epoch AI Jul 18, 2023
79 OLMo (7B) Ai2 29.5% 80 Exact match; 5-shot Official leaderboard via Epoch AI —

Best score over time

Each step is a new best result in our data, by evaluation date.

  1. 2023-06-1382.4%GPT-4 (Jun 2023)
  2. 2024-02-1582.7%Gemini 1.5 Pro (Feb 2024)
  3. 2024-02-2984.6%Claude 3 Opus
  4. 2024-06-2086.5%Claude 3.5 Sonnet
  5. 2024-09-2486.9%Gemini 1.5 Pro (Sept 2024)
  6. 2024-10-2287.3%Claude 3.5 Sonnet (October 2024)

Data sources & licences

SourceLicence / termsHow we reuse itLast fetched
Epoch AI Benchmarking Hub
Epoch AI, 'Capabilities & benchmarking', https://epoch.ai/benchmarks (CC BY 4.0)
 CC BY 4.0 Open licence – full data with attribution Oct 22, 2026
← All benchmarks · Learn how benchmarks work

FAQ

What does MMLU measure?

About 14,000 multiple-choice questions over 57 subjects, from elementary maths to law and medicine. Broad academic knowledge (usually 5-shot exact match). Data compiled by Epoch AI from papers, model reports and HELM.

How reliable is MMLU?

Saturated at the top; studies found a share of mislabelled questions; prompt format changes scores by several points. Contamination risk: High – Public since 2020 and widely copied on the web.

Data updated: Benchmarks(UTC+8)