MMLU-Pro Benchmark

MMLU-Pro Benchmark | Phaseo

Models Using This Benchmark

Organisation	Model	Reported	Top Score	Info	Self Reported	Source
Qwen	Qwen 3.6 Plus	01 Apr 2026	88.50%	-	Yes	Source
MiniMax	MiniMax M2.1	23 Dec 2025	88%	-	Yes	Source
Qwen	Qwen 3.5 397B A17B	16 Feb 2026	87.80%	-	Yes	Source
ByteDance	Seed 2.0 Lite	14 Feb 2026	87.70%	Seed2 official benchmark table \| MMLU-Pro	Yes	Source
DeepSeek	DeepSeek V4 Pro	24 Apr 2026	87.50%	EM; DeepSeek-V4-Pro Max	Yes	Source
Moonshot	Kimi K2.5	27 Jan 2026	87.10%	-	Yes	Source
Baidu	Ernie 5.0	22 Jan 2026	87%	-	Yes	Source
Baidu	Ernie 5.0 0110	-	87%	inferred version-family alias from ernie-5.0	Yes	Source
Baidu	Ernie 5.0 Preview 1203	-	87%	inferred version-family alias from ernie-5.0	Yes	Source
Baidu	Ernie 5.0 Preview 1220	-	87%	inferred version-family alias from ernie-5.0	Yes	Source
ByteDance	Seed 2.0 Pro	14 Feb 2026	87%	Seed2 official benchmark table \| MMLU-Pro	Yes	Source
Qwen	Qwen 3.5 122B A10B	24 Feb 2026	86.70%	-	Yes	Source
DeepSeek	DeepSeek V4 Flash	24 Apr 2026	86.40%	EM; DeepSeek-V4-Flash High	Yes	Source
Qwen	Qwen 3.5 27B	24 Feb 2026	86.10%	-	Yes	Source
Qwen	Qwen 3.5 Flash	23 Feb 2026	86.10%	inferred family alias from qwen3.5-27b (score=0.4147; benches=81)	Yes	Source
Qwen	Qwen 3.5 35B A3B	24 Feb 2026	85.30%	-	Yes	Source
Google	Gemma 4 31B	02 Apr 2026	85.20%	-	Yes	Source
DeepSeek	DeepSeek OCR 2	-	85%	inferred family alias from deepseek-v3.2-exp (score=0.3809; benches=14)	Yes	Source
DeepSeek	DeepSeek V3.2 Exp	29 Sept 2025	85%	-	Yes	Source
Xiaomi	MiMo V2 Flash	16 Dec 2025	84.90%	-	Yes	Source
Moonshot	Kimi K2 Thinking	06 Nov 2025	84.60%	inferred alias from kimi-k2-thinking-0905	Yes	Source
z.AI	GLM 4.5	28 Jul 2025	84.60%	-	Yes	Source
Qwen	Qwen 3 235B A22B Thinking 2507	-	84.40%	-	Yes	Source
z.AI	GLM 4.7	22 Dec 2025	84.30%	-	Yes	Source
LG	K EXAONE	31 Dec 2025	83.80%	inferred modality/version alias from k-exaone-236b-a23b	Yes	Source
Qwen	Qwen 3 VL 235B A22B Thinking	-	83.80%	-	Yes	-
Nvidia	Nemotron 3 Super	11 Mar 2026	83.73%	-	Yes	Source
DeepSeek	DeepSeek V3.1	21 Aug 2025	83.70%	Non-thinking: 83.7%, Thinking: 84.8%	Yes	Source
DeepSeek	DeepSeek V3.1 Terminus	22 Sept 2025	83.70%	inferred alias from deepseek-v3.1	Yes	Source
ByteDance	Seed 2.0 Mini	14 Feb 2026	83.60%	Seed2 official benchmark table \| MMLU-Pro	Yes	Source
Arcee AI	Trinity Large Thinking	01 Apr 2026	83.40%	Hugging Face model card benchmark table (arcee-ai/Trinity-Large-Thinking)	Yes	Source
SpaceXAI	Grok 3	18 Apr 2025	83.10%	-	Yes	Source
Qwen	Qwen 3 A235 A22B Instruct 2507	-	83%	-	Yes	Source
SpaceXAI	Grok 3 Mini	18 Apr 2025	82.80%	High Reasoning Effort	Yes	Source
Qwen	Qwen 3 Next 80B A3B Thinking	-	82.70%	-	Yes	Source
Meituan	LongCat Flash Chat	-	82.68%	inferred high-confidence family alias from longcat-flash-chat (score=0.4667; benches=16)	Yes	Source
Google	Gemma 4 26B A4B	02 Apr 2026	82.60%	-	Yes	Source
Qwen	Qwen 3.5 9B	02 Mar 2026	82.50%	-	Yes	Source
Qwen	Qwen 3 VL 32B Thinking	-	82.10%	-	Yes	-
MiniMax	MiniMax M2	27 Oct 2025	82%	-	Yes	-
MiniMax	MiniMax M2 Her	24 Jan 2026	82%	inferred modality/version alias from minimax-m2	Yes	-
LG	EXAONE 4.0 32B	15 Jul 2025	81.80%	Reasoning	Yes	Source
Qwen	Qwen 3 VL 235B A22B Instruct	-	81.80%	-	Yes	-
Amazon	Nova 2 Pro	02 Dec 2025	81.60%	-	Yes	Source
z.AI	GLM 4.5 Air	28 Jul 2025	81.40%	-	Yes	Source
MiniMax	MiniMax M1 80K	16 Jun 2025	81.10%	-	Yes	-
Moonshot	Kimi K2 (2025-09-05)	05 Sept 2025	81.10%	EM	Yes	Source
Amazon	Nova 2 Lite	02 Dec 2025	80.90%	-	Yes	Source
MiniMax	Minimax M1 40K	16 Jun 2025	80.60%	-	Yes	-
Qwen	Qwen 3 Next 80B A3B Instruct	-	80.60%	-	Yes	Source
Meta	Llama 4 Maverick	05 Apr 2025	80.50%	-	Yes	Source
Qwen	Qwen 3 VL 30B A3B Thinking	-	80.50%	-	Yes	-
SpaceXAI	Grok 3 Beta	19 Feb 2025	79.90%	-	Yes	Source
Qwen	Qwen 3.5 4B	02 Mar 2026	79.10%	-	Yes	Source
SpaceXAI	Grok 3 Mini Beta	19 Feb 2025	78.90%	-	Yes	Source
Qwen	Qwen 3 VL 32B Instruct	-	78.60%	-	Yes	-
Nvidia	Nemotron Nano 3 30B A3B	15 Dec 2025	78.30%	-	Yes	Source
Mistral	Mistral Small 1.0	26 Feb 2024	78%	inferred family alias from mistral-small-latest (score=0.3650; benches=9)	Yes	Source
Mistral	Mistral Small 2.0	17 Sept 2024	78%	inferred family alias from mistral-small-latest (score=0.3650; benches=9)	Yes	Source
Mistral	Mistral Small 4	16 Mar 2026	78%	-	Yes	Source
Mistral	Mistral Small Creative	16 Dec 2025	78%	inferred family alias from mistral-small-latest (score=0.4273; benches=9)	Yes	Source
Qwen	Qwen 3 VL 30B A3B Instruct	-	77.80%	-	Yes	-
Google	DiffusionGemma 26B A4B IT	10 Jun 2026	77.60%	-	Yes	Source
Qwen	Qwen 3 8B	-	77.30%	inferred high-confidence family alias from qwen3-vl-8b-thinking (score=0.4600; benches=50)	Yes	-
Qwen	Qwen 3 Embedding 8B	-	77.30%	inferred family alias from qwen3-vl-8b-thinking (score=0.3850; benches=50)	Yes	-
Qwen	Qwen 3 Guard Gen 8B	-	77.30%	inferred family alias from qwen3-vl-8b-thinking (score=0.3400; benches=50)	Yes	-
Qwen	Qwen 3 Guard Stream 8B	-	77.30%	inferred family alias from qwen3-vl-8b-thinking (score=0.3371; benches=50)	Yes	-
Qwen	Qwen 3 Reranker 8B	-	77.30%	inferred family alias from qwen3-vl-8b-thinking (score=0.3850; benches=50)	Yes	-
Qwen	Qwen 3 VL 8B Thinking	-	77.30%	-	Yes	-
Qwen	Qwen 3 VL Embedding 8B	-	77.30%	inferred high-confidence family alias from qwen3-vl-8b-thinking (score=0.5232; benches=50)	Yes	-
Qwen	Qwen 3 VL Reranker 8B	-	77.30%	inferred high-confidence family alias from qwen3-vl-8b-thinking (score=0.5275; benches=50)	Yes	-
Google	Gemma 4 12B	03 Jun 2026	77.20%	-	Yes	Source
Google	Gemini 2.0 Flash	05 Feb 2025	76.40%	-	Yes	Source
Google	Gemini 2.0 Flash Exp	-	76.40%	inferred alias from gemini-2.0-flash	Yes	Source
Google	Gemini 2.0 Pro Exp (2025-02-05)	05 Feb 2025	76.40%	inferred modality/version alias from gemini-2.0-flash	Yes	Source
Google	LearnLM 2.0 Flash Experimental	17 Apr 2025	76.40%	inferred family alias from gemini-2.0-flash (score=0.3700; benches=13)	Yes	Source
Microsoft	Phi 4 Reasoning Plus	30 Apr 2025	76%	-	Yes	Source
DeepSeek	DeepSeek OCR	20 Oct 2025	75.90%	inferred family alias from deepseek-v3 (score=0.3000; benches=20)	Yes	Source
DeepSeek	DeepSeek V2 (2024-06-28)	28 Jun 2024	75.90%	inferred family alias from deepseek-v3 (score=0.4159; benches=20)	Yes	Source
DeepSeek	DeepSeek V4	-	75.90%	inferred high-confidence family alias from deepseek-v3 (score=0.5818; benches=20)	Yes	Source
Google	Gemini 1.5 Pro 001	23 May 2024	75.80%	inferred alias from gemini-1.5-pro	Yes	Source
Google	Gemini 1.5 Pro Exp (2024-08-01)	01 Aug 2024	75.80%	inferred alias from gemini-1.5-pro	Yes	Source
Google	Gemini 1.5 Pro Exp (2024-08-27)	27 Aug 2024	75.80%	inferred alias from gemini-1.5-pro	Yes	Source
Google	Gemini Robotics ER 1.5 Preview	25 Sept 2025	75.80%	inferred family alias from gemini-1.5-pro (score=0.3717; benches=23)	Yes	Source
Google	LearnLM 1.5 Pro Experimental	19 Nov 2024	75.80%	inferred family alias from gemini-1.5-pro (score=0.3700; benches=23)	Yes	Source
SpaceXAI	Grok 2	13 Aug 2024	75.50%	-	Yes	Source
OpenAI	GPT 4o (2024-08-06)	06 Aug 2024	74.70%	-	Yes	Source
OpenAI	GPT 4o Audio (2025-06-03)	03 Jun 2025	74.70%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Audio (2024-10-01)	01 Oct 2024	74.70%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Audio (2024-12-17)	17 Dec 2024	74.70%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Realtime Preview (2025-06-03)	03 Jun 2025	74.70%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Realtime Preview (2024-10-01)	01 Oct 2024	74.70%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Search Preview	11 Mar 2025	74.70%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Transcribe	20 Mar 2025	74.70%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Transcribe Diarize	15 Oct 2025	74.70%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
Meta	Llama 4 Scout	05 Apr 2025	74.30%	-	Yes	Source
Microsoft	Phi 4 Reasoning	30 Apr 2025	74.30%	-	Yes	Source
Qwen	Qwen 3 4B Instruct 2507	-	73.60%	inferred family alias from qwen3-vl-4b-thinking (score=0.3462; benches=48)	Yes	-
Qwen	Qwen 3 4B SafeRL	-	73.60%	inferred family alias from qwen3-vl-4b-thinking (score=0.3850; benches=48)	Yes	-
Qwen	Qwen 3 4B Thinking 2507	-	73.60%	inferred family alias from qwen3-vl-4b-thinking (score=0.3462; benches=48)	Yes	-
Qwen	Qwen 3 Embedding 4B	-	73.60%	inferred family alias from qwen3-vl-4b-thinking (score=0.3850; benches=48)	Yes	-
Qwen	Qwen 3 Guard Gen 4B	-	73.60%	inferred family alias from qwen3-vl-4b-thinking (score=0.3400; benches=48)	Yes	-
Qwen	Qwen 3 Guard Stream 4B	-	73.60%	inferred family alias from qwen3-vl-4b-thinking (score=0.3371; benches=48)	Yes	-
Qwen	Qwen 3 Reranker 4B	-	73.60%	inferred family alias from qwen3-vl-4b-thinking (score=0.3850; benches=48)	Yes	-
Qwen	Qwen 3 VL 4B Thinking	-	73.60%	-	Yes	-
OpenAI	GPT 4o (2024-05-13)	13 May 2024	72.60%	-	Yes	Source
SpaceXAI	Grok 2 Mini	13 Aug 2024	72%	-	Yes	Source
Google	Gemini 2.0 Flash Lite	05 Feb 2025	71.60%	-	Yes	Source
Qwen	Qwen 3 VL 8B Instruct	-	71.60%	-	Yes	-
Qwen	Qwen 72B	-	71.10%	inferred family alias from qwen-2.5-72b-instruct (score=0.3060; benches=14)	Yes	Source
Microsoft	Phi 1	-	70.40%	inferred family alias from phi-4 (score=0.3100; benches=13)	Yes	Source
Microsoft	Phi 2	-	70.40%	inferred family alias from phi-4 (score=0.3100; benches=13)	Yes	Source
Microsoft	Phi 4	12 Dec 2024	70.40%	-	Yes	Source
Moonshot	Kimi K2 (2025-07-11)	11 Jul 2025	69.20%	EM	Yes	Source
Mistral	Mistral Small 3.2	20 Jun 2025	69.06%	5 Shot CoT	Yes	Source
Meta	Llama 2 70B Chat	20 Jun 2023	68.90%	inferred family alias from llama-3.3-70b-instruct (score=0.3129; benches=9)	Yes	Source
Qwen	Qwen 2.5 Coder 32B Instruct	-	68.80%	inferred high-confidence family alias from qwen2.5-vl-32b (score=0.4641; benches=28)	Yes	Source
Qwen	Qwen 2.5 VL 32B Instruct	24 Mar 2025	68.80%	-	Yes	Source
Qwen	Qwen 2.5 VL 3B Instruct	-	68.80%	inferred high-confidence family alias from qwen2.5-vl-32b (score=0.4914; benches=28)	Yes	Source
Qwen	Qwen 3 235B A22B	-	68.18%	-	Yes	Source
Google	Gemini 1.5 Flash 001	23 May 2024	67.30%	inferred alias from gemini-1.5-flash	Yes	Source
Google	Gemini 1.5 Flash Preview	14 May 2024	67.30%	inferred alias from gemini-1.5-flash	Yes	Source
Qwen	Qwen 3 VL 4B Instruct	-	67.10%	-	Yes	-
Qwen	Qwen 3.5 2B	02 Mar 2026	66.50%	-	Yes	Source
Qwen	Qwen 2 Math 72B	-	64.40%	inferred high-confidence family alias from qwen2-72b-instruct (score=0.4667; benches=17)	Yes	Source
Qwen	Qwen 2 Math RM 72B	-	64.40%	inferred family alias from qwen2-72b-instruct (score=0.3917; benches=17)	Yes	Source
IBM	Granite 4.1 30B	29 Apr 2026	64.09%	5-shot, CoT	Yes	Source
Qwen	Qwen 14B	-	63.70%	inferred family alias from qwen-2.5-14b-instruct (score=0.3060; benches=16)	Yes	Source
LG	EXAONE 4.0 1.2B	15 Jul 2025	59.30%	Reasoning	Yes	Source
Google	Gemini 1.5 Flash 8B	15 Mar 2024	58.70%	-	Yes	Source
Google	Gemini 1.5 Flash 8B Exp (2024-08-27)	27 Aug 2024	58.70%	inferred alias from gemini-1.5-flash-8b	Yes	Source
Google	Gemini 1.5 Flash 8B Exp (2024-09-24)	24 Sept 2024	58.70%	inferred alias from gemini-1.5-flash-8b	Yes	Source
Qwen	Qwen 7B	-	56.30%	inferred family alias from qwen-2.5-7b-instruct (score=0.3083; benches=14)	Yes	Source
IBM	Granite 4.1 8B	29 Apr 2026	55.99%	5-shot, CoT	Yes	Source
Microsoft	Phi 4 Mini	01 Feb 2025	52.80%	-	Yes	Source
SpaceXAI	Grok 1.5	28 Mar 2024	51%	-	Yes	Source
IBM	Granite 4.1 3B	29 Apr 2026	49.83%	5-shot, CoT	Yes	Source
Microsoft	Phi 3 Mini 128K Instruct	-	47.40%	inferred family alias from phi-3.5-mini-instruct (score=0.3533; benches=31)	Yes	Source
Microsoft	Phi 3.5 mini instruct	23 Aug 2024	47.40%	-	Yes	Source
Qwen	Qwen 2.5 Coder 3B	-	47%	inferred family alias from qwen2.5-omni-7b (score=0.3000; benches=45)	Yes	Source
Qwen	Qwen 2.5 Coder 7B	-	47%	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4700; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math 7B	-	47%	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4767; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math 7B PRM800K	-	47%	inferred family alias from qwen2.5-omni-7b (score=0.3696; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math PRM 7B	-	47%	inferred family alias from qwen2.5-omni-7b (score=0.4092; benches=45)	Yes	Source
Qwen	Qwen 2.5 Omni 3B	-	47%	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4933; benches=45)	Yes	Source
Qwen	Qwen 2.5 Omni 7B	-	47%	-	Yes	Source
Microsoft	Phi 3.5 MoE instruct	23 Aug 2024	45.30%	-	Yes	Source
Qwen	Qwen 2 Audio 7B	-	44.10%	inferred modality/version alias from qwen2-7b-instruct	Yes	Source
Qwen	Qwen 2 Math 7B	-	44.10%	inferred high-confidence family alias from qwen2-7b-instruct (score=0.4706; benches=14)	Yes	Source
Qwen	Qwen 3.5 0.8B	02 Mar 2026	42.30%	-	Yes	Source
Baidu	Ernie 4.5 21B A3B	-	16%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 21B A3B Thinking	-	16%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 300B A47B	-	16%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 Turbo	-	16%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 VL 28B A3B	-	16%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 VL 424B A47B	-	16%	inferred version-family alias from ernie-4.5	Yes	Source

Average Score

Score Range

Leading Model

MMLU-Pro

Recorded Results

Average Score

Score Range

Leading Model

Models Using This Benchmark