/

← All benchmarks

SWE-bench Pro

coding · repo-level-bugfix↗ Quelle

69Evidence

not yet measured

What it measures

Schwierigere, laengere Aufgaben als SWE-bench Verified, teils mit privatem Testsatz gegen Kontamination.

What it does not measure

Kurze Einzeldatei-Aenderungen.

How the score is composed

Currency50

Discrimination50

Headroom50

Contamination safety100

Gaming resistance100

Methodological rigor80

Reproducibility60

Details

Benchmaxxing risk: low
Difficulty: high
Public test set: no
Human-verified: yes
LLM-judged: no
Skala: 0–100 % solved (höher besser)

Hand-maintained — no automatic adapter yet.

Best measured models

No results ingested yet.