A new benchmark for models that have to get it right the first time. FORGE puts five hard problems in front of a model, one shot each, with no tools and no retries.
Dataset Card
A new benchmark for models that have to get it right the first time. FORGE puts five hard problems in front of a model, one shot each, with no tools and no retries. The model has to write code that holds up under load, enumerate every solution of a constraint puzzle, find the shortest plan in a closed world, trace an unfamiliar machine for hundreds of steps without dropping state, and name the true extremum of an uncertain system along with an input that reaches it. Each answer arrives as a single JSON object with a certificate attached, and the scoring code checks it against exhaustive search and executable witnesses. The scores are exact and they reproduce. Four of the five prompts also…
Excerpt from the card by Terablast Terion, licensed other.
Details
- Repository
- terion-mlx/forge
- Publisher
- Terablast Terion
- Task category
- Text generation
- Tags
- benchmark, evaluation, reasoning
- Size category
- n<1K
- Languages
- Not stated by the source
- Revision
- 2183e5e43356c858c257f94795ed0d227b2fa36a
- Last updated
- 2026-10-02
Files
17 files, 138.4 KB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| data/test.jsonl | Data | 32.9 KB | — |
| grader/cases.json | Data | 20.8 KB | — |
| grader/verification.json | Data | 1.1 KB | — |
| LICENSE | Documentation | 1.4 KB | — |
| NOTICE | Documentation | 209 B | — |
| README.md | Documentation | 4.7 KB | — |
| SUITE_README.md | Documentation | 6.7 KB | — |
| tests/test_01.md | Documentation | 9.5 KB | — |
| tests/test_02.md | Documentation | 5.1 KB | — |
| tests/test_03.md | Documentation | 4.2 KB | — |
| tests/test_04.md | Documentation | 7.0 KB | — |
| tests/test_05.md | Documentation | 3.7 KB | — |
| grader/grader.py | Other | 21.9 KB | — |
| grader/reference.py | Other | 2.8 KB | — |
| grader/selftest.py | Other | 12.2 KB | — |
| grader/worker.py | Other | 1.6 KB | — |
| .gitattributes | Repository | 2.5 KB | — |
License and Download
- License
- other
- Access
- Access requested at publisher
Terablast Terion grants access through its official repository on Hugging Face.