65Evidence
not yet measuredWhat it measures
Ob ein Agent Aufgaben in einer echten Terminalumgebung zu Ende bringt — Installieren, Bauen, Debuggen, Aufraeumen.
What it does not measure
Reine Codequalitaet, Sprachvielfalt.
How the score is composed
Currency50
Discrimination50
Headroom50
Contamination safety60
Gaming resistance100
Methodological rigor80
Reproducibility100
Details
- Benchmaxxing risk
- low
- Difficulty
- high
- Public test set
- yes
- Human-verified
- yes
- LLM-judged
- no
- Skala
- 0–100 % solved (höher besser)
Hand-maintained — no automatic adapter yet.
Best measured models
No results ingested yet.