Shingi 27B is a local decision model. You give it context (text, and optionally
images), a question and the possible answers. It returns a choice, calibrated probabilities or an ordinal
score. It reads the logit of every candidate answer directly and generates no
text.
The model is a single 7.2 GB ternary GGUF file (PQ2_0). It is
Bonsai 2 27B with
retrained block scales: the ternary weights are unchanged, only their scales
were trained. It needs no adapter.
Run it
On Linux with an NVIDIA GPU and the CUDA toolkit, or on a Mac with Apple Silicon:
curl -fsSL https://raw.githubusercontent.com/kortexa-ai/shingi-27b/main/run.sh | bash
This builds the pinned Prism runtime,
downloads this model into your Hugging Face cache and serves the API on
http://127.0.0.1:8765. Use --port to change the port. The code, API examples
and options are in kortexa-ai/shingi-27b.
Requirements
| Resource |
Need |
| GPU |
NVIDIA with at least 20 GB: RTX 4090, RTX PRO 6000 or DGX Spark (tested). The model uses about 8 GiB at the 16K context, about 9 GiB with image input. |
| System RAM |
about 1 GB for the server process, plus page cache for the 7.2 GB file |
| Disk |
7.2 GB for the weights, plus the runtime build |
| Mac |
Apple Silicon with 24 GB or more of unified memory (16 GB minimum), using Metal |
| Software |
Linux (x86-64 or aarch64) with the CUDA toolkit, or macOS with the Xcode command line tools; CMake and a C++17 compiler |
Speed
Median latency per decision, one request at a time, loading excluded:
| Request |
RTX PRO 6000 |
RTX 4090 |
DGX Spark |
| Short yes/no (~120 tokens) |
66 ms |
85 ms |
148 ms |
| Short choice, 3 options |
73 ms |
95 ms |
181 ms |
| Choice, 10 options (~1,300 tokens) |
246 ms |
306 ms |
802 ms |
| Long choice, 3 options (~3,000 tokens) |
881 ms |
1,045 ms |
2,984 ms |
| GPU memory in use |
8.3 GB |
8.2 GB |
7.9 GB |
With image input on the RTX 4090, one image and one question take about 0.66 s
(median) and the model uses about 9.2 GB.
Questions in one request share a single pass over the images and the state, so
extra questions are cheap: on an RTX PRO 6000, eight questions about one image take
about 1.8 s instead of 8.7 s.
Scores
External suites, canonical choice order, 16K context:
| Suite |
Items |
Accuracy |
| JevBench public |
231 |
85.7% |
| JevBench hard |
111 |
72.1% |
| DecisionBench |
586 |
75.3% |
| DecisionBench hard |
293 |
65.2% |
| This/That |
7,305 |
67.9% |
These suites also informed the choice of training data, so treat them as
development results rather than a clean held-out test.
More external suites. None of their items are in the training data, but Mind2Web test
shares websites with the Mind2Web train split we train on:
| Suite |
Metric |
Bonsai 2 27B (base) |
Shingi 27B |
Jev 1.13.0 (reported) |
| PubMedQA (1,000) |
accuracy |
72.1% |
78.8% |
78.7% |
| SummEval relevance |
Spearman |
0.563 |
0.534 |
0.526 |
| SummEval consistency |
Spearman |
0.537 |
0.550 |
0.555 |
| SummEval (both) |
score MAE vs experts |
0.94 |
0.51 |
— |
| RewardBench |
official score |
0.847 |
0.916 |
— |
| Open-Jev (4,204) |
accuracy |
66.7% |
85.0% |
— |
| SCOTUS (640, ≤40k chars) |
micro / macro-F1 |
66.1 / 57.3 |
64.4 / 56.4 |
72.6 / 62.1* |
| ECtHR A (300 cases) |
micro / macro-F1 |
49.2 / 39.5 |
57.9 / 56.8 |
— |
| ECtHR B (300 cases) |
micro / macro-F1 |
72.5 / 72.2 |
71.1 / 68.0 |
— |
| Mind2Web test |
element accuracy |
85.6% |
92.4% |
— |
The Jev numbers are third-party reports: PubMedQA and SummEval from
Deußer et al., arXiv 2609.37647 (same 1,000
PubMedQA items; summary-level Spearman), SCOTUS from the
chepyle/jev-test reanalysis.
*That SCOTUS run used the full test split with truncation; ours skips the 760 longest
opinions, so the two are not directly comparable.
Images
Shingi reads images through the Bonsai 2 27B vision projector (mmproj.gguf, included
here). Send up to 8 images per request: /v1/decisions follows SGLang's decision
endpoint, and /v1/systemone takes an optional images list.
Zero-shot:
| Suite |
Items |
Accuracy |
| VSR (spatial yes/no) |
300 |
82.0% |
| A-OKVQA (4-way) |
300 |
89.0% |
| VQAv2 yes/no |
300 |
89.0% |
The model was not trained on images; these come from the base model's vision with
Shingi's decision training on top.
Training data
Public datasets, used under their stated licenses:
Banking77,
CLINC150,
MMLU,
HelpSteer2,
HelpSteer,
Measuring Hate Speech,
Civil Comments,
GoEmotions,
LEDGAR (LexGLUE),
MASSIVE,
CommonsenseQA,
WinoGrande,
GSM8K,
HelpSteer3,
TabFact,
TAT-QA,
CaseHOLD and Unfair-ToS (LexGLUE),
TimeQA,
Mind2Web,
MedMCQA and
MedQA. Several
came through the jev-bench
repackaging.
About a third of the training tokens are model-generated: a private synthetic
dataset of decision tasks, plus the Apache-2.0 teacher data from
Mapika/decider.
Limitations
- English only.
- Much slower on a Mac than on an NVIDIA GPU: on an M4 Pro, a short decision takes about
1.2 s and a 1,100-token one about 12.6 s.
MLX was measured alongside Metal and was about 10% slower on the M4, so the
Mac build uses Metal.
License
Apache-2.0. Derived from Bonsai 2 27B by Prism ML (Apache-2.0), which descends
from Qwen3.8-27B; see NOTICE. The training datasets keep their own
licenses.