CharadesSTA

AI benchmark results and model performance

Type: percentageLanguage

Recorded Results

Average Score

57.09%

Score Range

43.60% - 64.80%

Leading Model (lowest score)

43.60% - Qwen 2.5 0.5B

Scores Over Time

Individual benchmark scores plotted by date.

Models Using This Benchmark

Organisation	Model	Reported	Top Score	Info	Self Reported	Source
Qwen	Qwen 2.5 0.5B	-	43.60%	inferred family alias from qwen2.5-vl-7b (score=0.3131; benches=32)	Yes	Source
Qwen	Qwen 2.5 1.5B	-	43.60%	inferred family alias from qwen2.5-vl-7b (score=0.3131; benches=32)	Yes	Source
Qwen	Qwen 2.5 3B	-	43.60%	inferred family alias from qwen2.5-vl-7b (score=0.3531; benches=32)	Yes	Source
Qwen	Qwen 2.5 VL 7B Instruct	-	43.60%	-	Yes	Source
Qwen	Qwen 2.5 Coder 32B Instruct	-	54.20%	inferred high-confidence family alias from qwen2.5-vl-32b (score=0.4641; benches=28)	Yes	Source
Qwen	Qwen 2.5 VL 32B Instruct	24 Mar 2025	54.20%	-	Yes	Source
Qwen	Qwen 2.5 VL 3B Instruct	-	54.20%	inferred high-confidence family alias from qwen2.5-vl-32b (score=0.4914; benches=28)	Yes	Source
Qwen	Qwen 3 VL 4B Instruct	-	55.50%	-	Yes	-
Qwen	Qwen 3 VL 8B Instruct	-	56%	-	Yes	-
Qwen	Qwen 3 4B Instruct 2507	-	59%	inferred family alias from qwen3-vl-4b-thinking (score=0.3462; benches=48)	Yes	-
Qwen	Qwen 3 4B SafeRL	-	59%	inferred family alias from qwen3-vl-4b-thinking (score=0.3850; benches=48)	Yes	-
Qwen	Qwen 3 4B Thinking 2507	-	59%	inferred family alias from qwen3-vl-4b-thinking (score=0.3462; benches=48)	Yes	-
Qwen	Qwen 3 Embedding 4B	-	59%	inferred family alias from qwen3-vl-4b-thinking (score=0.3850; benches=48)	Yes	-
Qwen	Qwen 3 Guard Gen 4B	-	59%	inferred family alias from qwen3-vl-4b-thinking (score=0.3400; benches=48)	Yes	-
Qwen	Qwen 3 Guard Stream 4B	-	59%	inferred family alias from qwen3-vl-4b-thinking (score=0.3371; benches=48)	Yes	-
Qwen	Qwen 3 Reranker 4B	-	59%	inferred family alias from qwen3-vl-4b-thinking (score=0.3850; benches=48)	Yes	-
Qwen	Qwen 3 VL 4B Thinking	-	59%	-	Yes	-
Qwen	Qwen 3 8B	-	59.90%	inferred high-confidence family alias from qwen3-vl-8b-thinking (score=0.4600; benches=50)	Yes	-
Qwen	Qwen 3 Guard Stream 8B	-	59.90%	inferred family alias from qwen3-vl-8b-thinking (score=0.3371; benches=50)	Yes	-
Qwen	Qwen 3 VL 8B Thinking	-	59.90%	-	Yes	-
Qwen	Qwen 3 VL Embedding 8B	-	59.90%	inferred high-confidence family alias from qwen3-vl-8b-thinking (score=0.5232; benches=50)	Yes	-
Qwen	Qwen 3 VL Reranker 8B	-	59.90%	inferred high-confidence family alias from qwen3-vl-8b-thinking (score=0.5275; benches=50)	Yes	-
Qwen	Qwen 3 VL 32B Instruct	-	61.20%	-	Yes	-
Qwen	Qwen 3 VL 30B A3B Thinking	-	62.70%	-	Yes	-
Qwen	Qwen 3 VL 32B Thinking	-	62.80%	-	Yes	-
Qwen	Qwen 3 VL 235B A22B Thinking	-	63.50%	-	Yes	-
Qwen	Qwen 3 VL 30B A3B Instruct	-	63.50%	-	Yes	-
Qwen	Qwen 3 VL 235B A22B Instruct	-	64.80%	-	Yes	-