MGSM

AI benchmark results and model performance

Type: percentageMath

Recorded Results

Average Score

79.05%

Score Range

47.90% - 92.30%

Leading Model (lowest score)

47.90% - Phi 3 Mini 128K Instruct

Scores Over Time

Individual benchmark scores plotted by date.

Models Using This Benchmark

Organisation	Model	Reported	Top Score	Info	Self Reported	Source
Microsoft	Phi 3 Mini 128K Instruct	-	47.90%	inferred family alias from phi-3.5-mini-instruct (score=0.3533; benches=31)	Yes	Source
Microsoft	Phi 3.5 mini instruct	23 Aug 2024	47.90%	-	Yes	Source
OpenAI	GPT 3.5 Turbo 16K 0613	13 Jun 2023	56.30%	inferred family alias from gpt-3.5-turbo-0125 (score=0.4244; benches=8)	No	Source
OpenAI	GPT 3.5 Turbo (2023-11-06)	28 Sept 2023	56.30%	inferred modality/version alias from gpt-3.5-turbo-0125	No	Source
OpenAI	GPT 3.5 Turbo Instruct	06 Nov 2023	56.30%	inferred high-confidence family alias from gpt-3.5-turbo-0125 (score=0.5539; benches=8)	No	Source
Microsoft	Phi 3.5 MoE instruct	23 Aug 2024	58.70%	-	Yes	Source
Microsoft	Phi 4 Mini	01 Feb 2025	63.90%	-	Yes	Source
Meta	Llama 3.2 11B Vision Instruct	-	68.90%	inferred high-confidence family alias from llama-3.2-11b-instruct (score=0.5873; benches=11)	Yes	Source
Meta	Llama 3.2 1B Instruct	25 Sept 2024	68.90%	inferred family alias from llama-3.2-11b-instruct (score=0.4200; benches=11)	Yes	Source
IBM	Granite 4.1 3B	29 Apr 2026	70%	8-shot	Yes	Source
IBM	Granite 4.1 30B	29 Apr 2026	71.12%	8-shot	Yes	Source
OpenAI	GPT 4 32K 0613	13 Jun 2023	74.50%	inferred high-confidence family alias from gpt-4-0613 (score=0.4899; benches=12)	Yes	Source
Microsoft	Phi 1	-	80.60%	inferred family alias from phi-4 (score=0.3100; benches=13)	Yes	Source
Microsoft	Phi 2	-	80.60%	inferred family alias from phi-4 (score=0.3100; benches=13)	Yes	Source
Microsoft	Phi 4	12 Dec 2024	80.60%	-	Yes	Source
IBM	Granite 4.1 8B	29 Apr 2026	82.32%	8-shot	Yes	Source
Google	Gemini 1.5 Flash 001	23 May 2024	82.60%	inferred alias from gemini-1.5-flash	Yes	Source
Google	Gemini 1.5 Flash Preview	14 May 2024	82.60%	inferred alias from gemini-1.5-flash	Yes	Source
Qwen	Qwen 3 235B A22B	-	83.53%	-	Yes	Source
Meta	Llama 3.2 90B Vision Instruct	-	86.90%	inferred high-confidence family alias from llama-3.2-90b-instruct (score=0.5873; benches=13)	Yes	Source
OpenAI	GPT 4o Mini Audio Preview	17 Dec 2024	87%	inferred modality/version alias from gpt-4o-mini-2024-07-18	Yes	Source
OpenAI	GPT 4o Mini Realtime Preview	17 Dec 2024	87%	inferred modality/version alias from gpt-4o-mini-2024-07-18	Yes	Source
OpenAI	GPT 4o Mini Search Preview	11 Mar 2025	87%	inferred modality/version alias from gpt-4o-mini-2024-07-18	Yes	Source
OpenAI	GPT 4o Mini Transcribe (2025-12-15)	15 Dec 2025	87%	inferred modality/version alias from gpt-4o-mini-2024-07-18	Yes	Source
OpenAI	GPT 4o Mini Transcribe (2025-03-20)	20 Mar 2025	87%	inferred modality/version alias from gpt-4o-mini-2024-07-18	Yes	Source
OpenAI	GPT 4o Mini TTS (2025-12-15)	15 Dec 2025	87%	inferred modality/version alias from gpt-4o-mini-2024-07-18	Yes	Source
OpenAI	GPT 4o Mini TTS (2025-03-20)	20 Mar 2025	87%	inferred modality/version alias from gpt-4o-mini-2024-07-18	Yes	Source
Google	Gemini 1.5 Pro 001	23 May 2024	87.50%	inferred alias from gemini-1.5-pro	Yes	Source
Google	Gemini 1.5 Pro Exp (2024-08-01)	01 Aug 2024	87.50%	inferred alias from gemini-1.5-pro	Yes	Source
Google	Gemini 1.5 Pro Exp (2024-08-27)	27 Aug 2024	87.50%	inferred alias from gemini-1.5-pro	Yes	Source
Google	Gemini Robotics ER 1.5 Preview	25 Sept 2025	87.50%	inferred family alias from gemini-1.5-pro (score=0.3717; benches=23)	Yes	Source
Google	LearnLM 1.5 Pro Experimental	19 Nov 2024	87.50%	inferred family alias from gemini-1.5-pro (score=0.3700; benches=23)	Yes	Source
OpenAI	GPT 4 Turbo Preview (1106)	06 Nov 2023	88.50%	inferred version-family alias from gpt-4-turbo-2024-04-09	Yes	Source
OpenAI	GPT 4o (2024-05-13)	13 May 2024	90.50%	-	Yes	Source
Meta	Llama 4 Scout	05 Apr 2025	90.60%	-	Yes	Source
OpenAI	o1 preview	12 Sept 2024	90.80%	-	Yes	Source
Meta	Llama 2 70B Chat	20 Jun 2023	91.10%	inferred family alias from llama-3.3-70b-instruct (score=0.3129; benches=9)	Yes	Source
OpenAI	o3 mini	30 Jan 2025	92%	-	Yes	Source
Meta	Llama 4 Maverick	05 Apr 2025	92.30%	-	Yes	Source