Darwin-27B-ZTC-v9 is a decision model. It does not generate text. Given a state and a typed question, it returns a calibrated probability for every option in one forward pass: yes or no (noul), one of several options (choice), or a level on an ordered scale (score). Requests use the System One shape (state plus questions), so clients written for that format work unchanged.
It is the next version of Darwin-27B-ZTC-v5. The main change is much stronger multiple-choice decisions (intent, entity type, relation, part of speech, rule reasoning, product relevance) while keeping the yes/no and score behaviour of v5.
Results
S1MB (System One Mosaic Benchmark), english-v1
All 137 benchmarks complete, measured with the S1MB evaluator and its unchanged autojev adapter, zero failed decisions. The ranking follows the leaderboard's own Borda method (viewer/src/lib/borda.ts) applied to the published results (171 complete models, 2026-10-17) with this model added. Submission pending maintainer review; see the leaderboard.
| Rank |
Model |
Borda score |
Mean baseline-adjusted score |
| 1 |
Darwin-27B-ZTC-v9 |
91.29 |
73.95 |
| 2 |
Sqwish kettei-fat v1.0 (API) |
91.11 |
73.54 |
| 3 |
Punky Tiger Labs Superion S1 |
90.58 |
76.51 |
| 4 |
FINAL-Bench/Darwin-27B-ZTC-v5 |
89.57 |
72.07 |
| 5 |
perplexity-ai/pplx-decider-v1.1-27b |
87.14 |
71.56 |
| 6 |
torchcast-ai/torchcast-decision-27b |
86.93 |
69.69 |
Largest gains over v5 (baseline-adjusted score)
| Benchmark |
v5 |
v9 |
| boardgameqa-choice |
0.500 |
0.953 |
| ud-ewt-choice (part of speech) |
0.433 |
0.833 |
| gretel-pii-choice (entity type) |
0.653 |
0.981 |
| wiqa-choice |
0.365 |
0.667 |
| fol-nli-choice |
0.279 |
0.574 |
| snli-choice |
0.790 |
0.968 |
| esci-choice |
0.315 |
0.466 |
| argument-quality-choice |
0.857 |
0.959 |
Training and data
Full-weight fine-tuning in the Darwin-27B-ZTC line on public train splits of decision datasets under licenses that allow commercial use. For v9 we added the original upstream train splits of choice-task sources (for example DBpedia-14, SNLI, MASSIVE, HWU64, BANKING77, CLINC150, MTOP, Few-NERD, UD English-EWT, Gretel PII masking, ESCI, WIQA, FOL-NLI, BoardgameQA, IBM argument quality), converted to the System One format.
- No test split of S1MB or of any benchmark was used for training.
- Every training row was screened against all S1MB cases (exact match of the normalised state, rare long text fields and duplicate states); matches were removed. Converters were checked by reproducing the S1MB test cases from the upstream test splits, and those test splits were used only for that check.
- Inference is one forward pass per question with the checkpoint temperature; no ensembling.
Usage
Python 3.12, a CUDA GPU with about 60 GB of free memory (BF16 weights), and the code shipped in this repository (autojev/).
import sys
from pathlib import Path
from huggingface_hub import snapshot_download
path = Path(snapshot_download("FINAL-Bench/Darwin-27B-ZTC-v9"))
sys.path.insert(0, str(path))
from autojev.model import DecisionModel, answer
model = DecisionModel(checkpoint=path).eval()
question = {
"type": "choice",
"instructions": "Identify the banking intent expressed by the utterance.",
"criteria": {"card_arrival": "Card delivery timing", "lost_or_stolen_card": "Card lost or stolen", "top_up_failed": "Top-up failed"},
}
row = {"state": {"utterance": "How long does a new card take to arrive?"}, "question": question}
print(answer(question, model.predict([row])[0]))
ztc_server.py serves POST /v1/systemone with batching.
curl http://localhost:8000/v1/systemone -H "Content-Type: application/json" -d '{"model": "FINAL-Bench/Darwin-27B-ZTC-v9",
"state": "I was charged twice for my order and want a refund.",
"questions": {
"refund": {"type": "noul", "instructions": "Is a refund requested?"},
"intent": {"type": "choice", "instructions": "What does the customer want?",
"criteria": {"refund": "wants money back", "cancel": "wants to cancel an order", "track": "wants to track an order"}}}}'
License
Apache-2.0. The decision code in autojev/ is MIT (upstream autojev by Denis Yarats) with changes for text-only backbones. Training data keep their own licenses; several sources are CC BY-SA (DBpedia-14, SNLI, UD English-EWT, MTOP, Few-NERD) and are credited here.
