HellaSwag

AI benchmark results and model performance

Type: percentageReasoning

Recorded Results

Average Score

71.65%

Score Range

33% - 95.30%

Leading Model (lowest score)

33% - Ernie 4.5 21B A3B

Scores Over Time

Individual benchmark scores plotted by date.

Models Using This Benchmark

Organisation	Model	Reported	Top Score	Info	Self Reported	Source
Baidu	Ernie 4.5 21B A3B	-	33%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 21B A3B Thinking	-	33%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 300B A47B	-	33%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 Turbo	-	33%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 VL 28B A3B	-	33%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 VL 424B A47B	-	33%	inferred version-family alias from ernie-4.5	Yes	Source
Microsoft	Phi 4 Mini	01 Feb 2025	69.10%	-	Yes	Source
Microsoft	Phi 3 Mini 128K Instruct	-	69.40%	inferred family alias from phi-3.5-mini-instruct (score=0.3533; benches=31)	Yes	Source
Microsoft	Phi 3.5 mini instruct	23 Aug 2024	69.40%	-	Yes	Source
Google	Gemma 3n E2B	25 Jun 2025	72.20%	-	Yes	Source
Google	Gemma 3n E4B	25 Jun 2025	78.60%	-	Yes	Source
Mistral	Mistral Nemo 12B	18 Jul 2024	83.50%	inferred family alias from mistral-nemo-instruct-2407 (score=0.3250; benches=8)	Yes	Source
Microsoft	Phi 3.5 MoE instruct	23 Aug 2024	83.80%	-	Yes	Source
Nvidia	Llama 3.1 Nemotron 70B Instruct	01 Oct 2024	85.58%	-	Yes	Source
Google	Gemini 1.5 Flash 001	23 May 2024	86.50%	inferred alias from gemini-1.5-flash	Yes	Source
Google	Gemini 1.5 Flash Preview	14 May 2024	86.50%	inferred alias from gemini-1.5-flash	Yes	Source
Qwen	Qwen 2 Math 72B	-	87.60%	inferred high-confidence family alias from qwen2-72b-instruct (score=0.4667; benches=17)	Yes	Source
Qwen	Qwen 2 Math RM 72B	-	87.60%	inferred family alias from qwen2-72b-instruct (score=0.3917; benches=17)	Yes	Source
Google	Gemini 1.5 Pro 001	23 May 2024	93.30%	inferred alias from gemini-1.5-pro	Yes	Source
Google	Gemini 1.5 Pro Exp (2024-08-01)	01 Aug 2024	93.30%	inferred alias from gemini-1.5-pro	Yes	Source
Google	Gemini 1.5 Pro Exp (2024-08-27)	27 Aug 2024	93.30%	inferred alias from gemini-1.5-pro	Yes	Source
Google	Gemini Robotics ER 1.5 Preview	25 Sept 2025	93.30%	inferred family alias from gemini-1.5-pro (score=0.3717; benches=23)	Yes	Source
Google	LearnLM 1.5 Pro Experimental	19 Nov 2024	93.30%	inferred family alias from gemini-1.5-pro (score=0.3700; benches=23)	Yes	Source
OpenAI	GPT 4 32K 0613	13 Jun 2023	95.30%	inferred high-confidence family alias from gpt-4-0613 (score=0.4899; benches=12)	Yes	Source