decider-2b

decider-2b (Verdict-ready)

The decider-2b System One decision model (Qwen3.5-2B-Base, 1.9B parameters, 32768-token context, checkpoint revision v11 — the vendor's default), packaged for Verdict — a high-performance serving engine for System One decision models. Serve it with one command:

This package serves through the torch runtime, which runs on CUDA only in v0.2, so there is one image and it needs a GPU. The -v mounts cache the checkpoint, so only the first run downloads it.

# GPU host (no CPU image: the torch runtime serves on CUDA in v0.2)
docker run --rm --gpus all -p 8080:8080 \
    -v ~/.cache/verdict:/root/.cache/verdict \
    sgyaqing/verdict:0.2.0-cuda serve verdict-models/decider-2b --host 0.0.0.0

If the hub is slow or unreachable, add -e HF_ENDPOINT=https://hf-mirror.com — the mirror covers both the package and the upstream checkpoint, since both are fetched over the Hugging Face path. Add -e VERDICT_USE_MODELSCOPE=1 and use sgyaqing/decider-2b to fetch from ModelScope instead; the upstream checkpoint is looked up there under the id the manifest carries (Mapika/decider-2b), which is the same id it has on Hugging Face.

Endpoints: the standard decision protocols POST /v1/systemone and POST /v1/decisions, native POST /v1/choice | /v1/score | /v1/noul, plus an OpenAI-compatible layer (/v1/chat/completions, /v1/models).

Runtime: torch (bf16)

This package serves through Verdict's torch runtime: row building, slot selection, calibration and the HTTP protocol run in the Rust engine; the model's forward pass runs in a pinned python torch stack (torch 2.14.1+cu130, transformers 5.19.0, flash-linear-attention 0.5.2) executing the official checkpoint in bf16 — the checkpoint's own precision, matching the vendor's own GPU deployment (dtype=bfloat16).

The weights are not redistributed in this repository. The manifest names the official source, and the downloader fetches them on first serve:

"checkpoint": { "repo": "Mapika/decider-2b",
                "files": ["config.json", "model.safetensors", "generation_config.json"] }

Files

Requirements: a CUDA GPU (the torch runtime serves on CUDA in v0.2), python3.12 with the pinned stack (the Verdict CUDA image ships it), ~5.5 GB VRAM at load (measured: 5654 MiB in the shipped image).

Verified against the official model

The official forward pass (decider-ai 1.9.0, CPU fp32) anchors a 30-case golden corpus (choice / score / noul, English, mixed lengths up to 2063 tokens). This package's serving path is graded against it end to end in the shipped configuration (torch runtime, bf16, torch.compile + CUDA graphs, RTX 4090): argmax flips: 0.

Performance

Measured on one machine (RTX 4090, AMD EPYC 7543, 16 threads), end to end over HTTP with oha and an identical request body for both servers. The baseline is the model author's own server (decider.serve); the Verdict column is this package. Latencies are per-request P50 / P99 at the stated concurrency. The ratios are oriented so that above 1× means Verdict is ahead in both: throughput ratio = Verdict ÷ baseline, latency ratio = baseline ÷ Verdict.

Both the accuracy and the numbers below are from the vendor's v11 weights — what Mapika/decider-2b serves today. The manifest names the repository rather than a revision, so a future upstream release would be picked up on a fresh download rather than silently graded against these figures.

License

Apache-2.0, inherited from the base checkpoint. Credit for the model itself belongs to the original author and the Qwen3.5-2B-Base it builds on; this repository only packages the serving manifest.