/

Benchmarks

Not just the largest pool, but a rated one. Every benchmark is scored on how current, discriminative and gameable it is — and that score is the weight it carries in every ranking here.

74 Benchmarks18 domains431 Results

coding (12)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
LiveCodeBenchcompetitive-programming	80	active	low	4
SWE-bench Verifiedrepo-level-bugfix	71	active	medium	6
SWE-bench Prorepo-level-bugfix	69	not yet measured	low	0
SWE-Lancereconomic-value	67	not yet measured	low	0
BigCodeBenchlibrary-use	66	active	low	2
Terminal-Benchshell-agent	65	not yet measured	low	0
SWE-bench Multilingualrepo-level-bugfix	58	not yet measured	low	0
HumanEvalfunction-synthesis	57	saturated	medium	5
CRUXEvalcode-reasoning	48	not yet measured	medium	0
RepoBenchrepo-completion	46	not yet measured	medium	0
Aider Polyglotedit-format	42	outdated	medium	0
MBPP+function-synthesis	40	not yet measured	medium	0

vision (7)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
MMMU-Proexpert-multimodal	80	active	medium	4
DocVQAdocument-understanding	71	saturated	low	3
BLINKvisual-perception	63	not yet measured	low	0
MathVistavisual-math	53	not yet measured	low	1
RealWorldQAspatial-understanding	44	not yet measured	medium	0
AI2Ddiagram-understanding	44	active	medium	2
ChartQAchart-reading	41	active	medium	2

audio-stt (7)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
Earnings-22domain-specific	68	active	low	21
AMI Meeting Corpusmeetings	63	not yet measured	low	0
Open ASR Leaderboardaggregate-wer	58	active	low	21
Common Voice (ASR)crowdsourced-accents	52	saturated	low	127
FLEURSmultilingual-asr	52	saturated	low	159
TED-LIUM 3prepared-speech	49	saturated	low	21
LibriSpeech test-otherread-speech	48	not yet measured	low	0

agentic (6)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
GAIAgeneral-assistant	67	not yet measured	low	0
BrowseCompweb-research	67	not yet measured	low	0
OSWorldcomputer-use	63	not yet measured	low	0
TAU-benchcustomer-workflows	55	not yet measured	medium	0
WebArenaweb-navigation	49	not yet measured	medium	0
MLE-benchml-engineering	45	not yet measured	medium	0

reasoning (6)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
ARC-AGI-2abstract-reasoning	71	not yet measured	low	0
Humanity's Last Examexpert-frontier	67	not yet measured	low	0
ZebraLogicconstraint-logic	53	not yet measured	medium	0
GPQA Diamondexpert-science	50	saturated	high	14
MuSRmulti-step-narrative	46	not yet measured	medium	0
BIG-Bench Hardmixed	40	not yet measured	medium	0

safety (5)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
JailbreakBenchadaptive-attacks	63	not yet measured	low	0
HarmBenchjailbreak-robustness	55	not yet measured	low	0
AgentHarmagentic-misuse	53	not yet measured	low	0
XSTestover-refusal	51	not yet measured	low	0
TruthfulQAfactuality	41	not yet measured	medium	0

long-context (5)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
LongBench v2realistic-documents	90	active	low	3
RULERsynthetic-retrieval	68	saturated	medium	2
MRCRmulti-round-coreference	53	not yet measured	medium	0
BABILongreasoning-in-haystack	53	not yet measured	medium	0
Needle in a Haystackretrieval	46	not yet measured	medium	0

instruction-following (5)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
LMArenahuman-preference	63	not yet measured	low	0
IFEvalverifiable-constraints	45	not yet measured	medium	0
Arena-Hardllm-judged-preference	43	not yet measured	medium	0
WildBenchreal-user-tasks	43	not yet measured	medium	0
AlpacaEval 2.0llm-judged-preference	39	not yet measured	medium	0

math (4)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
FrontierMathresearch-level	67	not yet measured	low	0
HMMTcompetition	61	saturated	medium	6
AIMEcompetition	53	saturated	medium	10
MATH-500school-competition	35	saturated	medium	1

multilingual (4)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
INCLUDEregional-knowledge	57	not yet measured	low	0
Global-MMLUknowledge	49	not yet measured	medium	0
Belebelereading-comprehension	49	not yet measured	medium	0
MGSMmath-transfer	44	active	medium	2

knowledge (3)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
SimpleQAfactuality	81	active	medium	2
MMLU-Probroad-academic	50	not yet measured	medium	0
MMLUbroad-academic	43	superseded	high	11

efficiency (3)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
Ausgabegeschwindigkeit (Token/s)latency	56	not yet measured	low	0
Zeit bis zum ersten Tokenlatency	56	not yet measured	low	0
Artificial Analysis Intelligence Indexaggregate-index	53	not yet measured	low	0

retrieval (2)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
AIR-Benchfresh-retrieval	62	not yet measured	low	0
BEIRzero-shot-retrieval	42	not yet measured	medium	0

tool-use (1)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
Berkeley Function Calling Leaderboardfunction-calling	55	not yet measured	low	0

translation (1)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
FLORES-200machine-translation	55	not yet measured	low	0

multimodal (1)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
Video-MMEvideo-understanding	81	active	low	2

audio-tts (1)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
TTS Arenahuman-preference	63	not yet measured	low	0

embedding (1)

Benchmark	Evidence	Status	Benchmaxxing risk	Results
MTEBaggregate	42	not yet measured	medium	0