MMMT Bench

AI benchmark results and model performance

Type: percentage

Recorded Results

Average Score

17.50%

Score Range

6% - 84.90%

Leading Model (lowest score)

6% - Qwen 2.5 Coder 3B

Scores Over Time

Individual benchmark scores plotted by date.

Models Using This Benchmark

Organisation	Model	Reported	Top Score	Info	Self Reported	Source
Qwen	Qwen 2.5 Coder 3B	-	6%	inferred family alias from qwen2.5-omni-7b (score=0.3000; benches=45)	Yes	Source
Qwen	Qwen 2.5 Coder 7B	-	6%	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4700; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math 7B	-	6%	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4767; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math 7B PRM800K	-	6%	inferred family alias from qwen2.5-omni-7b (score=0.3696; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math PRM 7B	-	6%	inferred family alias from qwen2.5-omni-7b (score=0.4092; benches=45)	Yes	Source
Qwen	Qwen 2.5 Omni 3B	-	6%	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4933; benches=45)	Yes	Source
Qwen	Qwen 2.5 Omni 7B	-	6%	-	Yes	Source
Qwen	Qwen 3 VL 4B Instruct	-	7.50%	-	Yes	-
Qwen	Qwen 3 4B Instruct 2507	-	7.70%	inferred family alias from qwen3-vl-4b-thinking (score=0.3462; benches=48)	Yes	-
Qwen	Qwen 3 4B SafeRL	-	7.70%	inferred family alias from qwen3-vl-4b-thinking (score=0.3850; benches=48)	Yes	-
Qwen	Qwen 3 4B Thinking 2507	-	7.70%	inferred family alias from qwen3-vl-4b-thinking (score=0.3462; benches=48)	Yes	-
Qwen	Qwen 3 Embedding 4B	-	7.70%	inferred family alias from qwen3-vl-4b-thinking (score=0.3850; benches=48)	Yes	-
Qwen	Qwen 3 Guard Gen 4B	-	7.70%	inferred family alias from qwen3-vl-4b-thinking (score=0.3400; benches=48)	Yes	-
Qwen	Qwen 3 Guard Stream 4B	-	7.70%	inferred family alias from qwen3-vl-4b-thinking (score=0.3371; benches=48)	Yes	-
Qwen	Qwen 3 Reranker 4B	-	7.70%	inferred family alias from qwen3-vl-4b-thinking (score=0.3850; benches=48)	Yes	-
Qwen	Qwen 3 VL 4B Thinking	-	7.70%	-	Yes	-
Qwen	Qwen 3 VL 8B Instruct	-	7.70%	-	Yes	-
Qwen	Qwen 3 VL 30B A3B Thinking	-	7.90%	-	Yes	-
Qwen	Qwen 3 8B	-	8%	inferred high-confidence family alias from qwen3-vl-8b-thinking (score=0.4600; benches=50)	Yes	-
Qwen	Qwen 3 Guard Stream 8B	-	8%	inferred family alias from qwen3-vl-8b-thinking (score=0.3371; benches=50)	Yes	-
Qwen	Qwen 3 VL 8B Thinking	-	8%	-	Yes	-
Qwen	Qwen 3 VL Embedding 8B	-	8%	inferred high-confidence family alias from qwen3-vl-8b-thinking (score=0.5232; benches=50)	Yes	-
Qwen	Qwen 3 VL Reranker 8B	-	8%	inferred high-confidence family alias from qwen3-vl-8b-thinking (score=0.5275; benches=50)	Yes	-
Qwen	Qwen 3 VL 30B A3B Instruct	-	8.10%	-	Yes	-
Qwen	Qwen 3 VL 32B Thinking	-	8.30%	-	Yes	-
Qwen	Qwen 3 VL 32B Instruct	-	8.40%	-	Yes	-
Qwen	Qwen 3 VL 235B A22B Instruct	-	8.50%	-	Yes	-
Qwen	Qwen 3 VL 235B A22B Thinking	-	8.50%	-	Yes	-
Mistral	Pixtral 12B	17 Sept 2024	60.50%	inferred version-family alias from pixtral-12b-2409	Yes	Source
Mistral	Pixtral Large	18 Nov 2024	74%	-	Yes	Source
Mistral	Ministral 3.0 3B	02 Dec 2025	78.30%	Multimodal	Yes	Source
Mistral	Ministral 3.0 8B	02 Dec 2025	80.80%	Multimodal	Yes	Source
Mistral	Ministral 3.0 14B	02 Dec 2025	84.90%	Multimodal	Yes	Source