SWE Bench Pro

AI benchmark results and model performance

Type: percentage

Recorded Results

Average Score

58.65%

Score Range

46% - 80.30%

Leading Model (lowest score)

46% - Seed 2.0 Lite

Scores Over Time

Individual benchmark scores plotted by date.

Models Using This Benchmark

Organisation	Model	Reported	Top Score	Info	Self Reported	Source
ByteDance	Seed 2.0 Lite	14 Feb 2026	46%	Seed2 official benchmark table \| SWE-Bench Pro	Yes	Source
Poolside	Laguna XS.2	28 Apr 2026	46.30%	public dataset mean pass@1	Yes	Source
ByteDance	Seed 2.0 Pro	14 Feb 2026	46.90%	Seed2 official benchmark table \| SWE-Bench Pro	Yes	Source
Poolside	Laguna XS 2.1	02 Jul 2026	47.60%	public dataset mean pass@1	Yes	Source
Moonshot	Kimi K2.5	27 Jan 2026	50.70%	-	Yes	Source
Meta	Muse Glimmer 30B	10 Aug 2026	51.20%	High reasoning	Yes	Source
Anthropic	Claude Opus 4.5	24 Nov 2025	52%	Avg@5	Yes	Source
Meta	Muse Spark	08 Apr 2026	52.40%	-	Yes	Source
OpenAI	GPT 5.4 Nano	17 Mar 2026	52.40%	Public	Yes	Source
DeepSeek	DeepSeek V4 Flash	24 Apr 2026	52.60%	Resolved; DeepSeek-V4-Flash Max	Yes	Source
Google	Gemini 3.1 Pro Preview	19 Feb 2026	54.20%	Single Attempt	Yes	-
OpenAI	GPT 5.4 Mini	17 Mar 2026	54.40%	Public	Yes	Source
DeepSeek	DeepSeek V4 Pro	24 Apr 2026	55.40%	Resolved; DeepSeek-V4-Pro Max	Yes	Source
MiniMax	MiniMax M2.5	12 Feb 2026	55.40%	-	Yes	Source
OpenAI	GPT 5.2	11 Dec 2025	55.60%	-	Yes	Source
Xiaomi	MiMo V2.5	22 Apr 2026	56.10%	Hugging Face evaluation result	Yes	Source
MiniMax	MiniMax M2.7	18 Mar 2026	56.20%	-	Yes	Source
OpenAI	GPT 5.2 Codex	18 Dec 2025	56.40%	-	Yes	Source
Qwen	Qwen 3.6 Plus	01 Apr 2026	56.60%	-	Yes	Source
OpenAI	GPT 5.3 Codex	05 Feb 2026	56.80%	Public; xhigh reasoning	Yes	Source
OpenAI	GPT 5 Pro	07 Aug 2025	57.70%	inferred family alias from gpt-5.4 (score=0.4083; benches=19)	Yes	Source
OpenAI	GPT 5 Search API	14 Oct 2025	57.70%	inferred family alias from gpt-5.4 (score=0.3050; benches=19)	Yes	Source
OpenAI	GPT 5.4	05 Mar 2026	57.70%	Public	Yes	Source
z.AI	GLM 5.1	07 Apr 2026	58.40%	-	Yes	Source
OpenAI	GPT 5.5	23 Apr 2026	58.60%	Public	Yes	Source
MiniMax	MiniMax M3	01 Jun 2026	59%	-	Yes	Source
Sakana AI	Fugu	22 Jun 2026	59%	Sakana Fugu product page benchmark table.	Yes	Source
Poolside	Laguna S 2.1	21 Jul 2026	59.40%	public dataset mean pass@1	Yes	Source
Qwen	Qwen 3.7 Max	21 May 2026	60.60%	SWE-Pro; refined benchmark; internal agent scaffold	Yes	Source
Meta	Muse Spark 1.1	09 Jul 2026	61.50%	Diverse software engineering	Yes	Source
Qwen	Qwen 3.8 27B	14 Aug 2026	61.70%	-	Yes	Source
Anthropic	Claude Sonnet 5	30 Jun 2026	63.20%	SWE-bench Pro; standard configuration; average over five trials	Yes	Source
Anthropic	Claude Opus 4.7	16 Apr 2026	64.30%	-	Yes	Source
Anthropic	Claude Opus 4.8	28 May 2026	69.20%	Max effort; avg 5 trials	Yes	Source
Sakana AI	Fugu Ultra	22 Jun 2026	73.70%	Sakana Fugu product page benchmark table.	Yes	Source
Anthropic	Claude Mythos Preview	07 Apr 2026	77.80%	-	Yes	Source
Anthropic	Claude Opus 5	24 Jul 2026	79.20%	Adaptive thinking at max effort; default sampling settings; average of 5 trials	Yes	Source
Anthropic	Claude Fable 5	09 Jun 2026	80.30%	Higher of Mythos 5 / Fable 5 in Anthropic's launch table	Yes	Source
Anthropic	Claude Mythos 5	09 Jun 2026	80.30%	Higher of Mythos 5 / Fable 5 in Anthropic's launch table	Yes	Source