Benchmarking Persistent Operational Agency in Source-Grounded Executable Systems This is the current public dataset for OPERATE. Filenames are versionless; immutable Hugging Face commits identify each snapshot.
Dataset Card
Benchmarking Persistent Operational Agency in Source-Grounded Executable Systems This is the current public dataset for OPERATE. Filenames are versionless; immutable Hugging Face commits identify each snapshot. The matching public code lives at Xnhyacinth/OPERATE. The current catalog contains 769 Full scenarios across 502 physical sources and 141 Lite scenarios across 82 sources. Qualification scoring is 0.15.0. Live evaluations use scoring 0.21.0 (waitrelativeoutcomev1). Live realtime uses nativedtv1; the default speed card is 37 Core rows whose native tick is at most 60s and whose plant wall is at most 30 minutes. logicalpersistent is the primary treatment. realtimepersistent is a…
Excerpt from the card by Huanxuan Liao, licensed other.
Structure
full 769 rows
| Split | Rows | Size |
|---|---|---|
| test | 769 | 8.7 MB |
lite 141 rows
| Split | Rows | Size |
|---|---|---|
| test | 141 | 1.7 MB |
Details
- Repository
- Xnhyacinth/OPERATE
- Publisher
- Huanxuan Liao
- Task category
- Other
- Tags
- benchmark, llm-agents, agentic-evaluation
- Size category
- Not stated by the source
- Languages
- llm-agents
- Revision
- 6904606eddda2cee51c4db319ac1bc28b4509b0e
- Last updated
- 2026-10-03
Files
12 files, 566.5 MB in total.
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| MANIFEST.json | Data | 1.4 MB | — |
| backend_runtime_closure.json | Data | 461.3 KB | — |
| backends.tar.zst | Data | 292.9 MB | a5ea0ea4ec3d |
| candidate_closure.json | Data | 1.8 KB | — |
| full/test-00000-of-00001.parquet | Data | 639.4 KB | 1dd54dcaafdd |
| lite/test-00000-of-00001.parquet | Data | 1.4 MB | 80f1c6f7d0aa |
| operate_v0_62_0_backends.tar.zst | Data | 269.2 MB | 4da05edddf6d |
| operate_v0_62_0_formal_evidence.tar.zst | Data | 457.3 KB | ec6547fc76e9 |
| parquet_manifest.json | Data | 731 B | — |
| release_manifest.json | Data | 31.2 KB | — |
| README.md | Documentation | 2.3 KB | — |
| .gitattributes | Repository | 2.5 KB | — |
License and Download
- License
- other
- Access
- No access gate
Released by Huanxuan Liao through its official repository on Hugging Face.