JEV-27B-VL-AWQ

4-bit AWQ quantization of autotrust/JEV-27B-VL, built with llm-compressor and served with vLLM. System 1 (POST /v1/decide) runs on a single 24 GB GPU — the fp8 build needs two — and on every axis we measured it is faster than the two-GPU fp8 deployment it replaces.

The System 1 adapter is merged into the weights (including the decision head), so no --enable-lora is needed. The decision head (lm_head) and the vision tower are kept in bf16; every other text layer is quantized, including the Gated DeltaNet projections of the 48 linear-attention layers — most Qwen3.5-family quants exclude those and therefore do not fit on one 24 GB card.

What is in this repo

Quantization

The AWQ step matters: a data-free RTN build of the same scheme measured KL 0.0144 vs the teacher and a 0.0019 card-example diff; the AWQ smoothing improves both (0.0122 at 200 rows, 0.0005–0.0006 card diff).

Validation

Protocol: the base model's own example decision (the billing / shipping / tech support prompt from the model card) plus 500 held-out rows sampled from the test split of jev-distill-corpus-v3 (rows with 2–16 options, all three kinds), scored against the target distributions in the corpus (the teacher's calibrated probabilities). Raw files: eval/.

Two honest notes on that last row. First, 47% of those 200 rows have flat teacher distributions (max probability < 0.6, mean 0.66): on them the teacher's top option is close to a coin flip, so top-1 agreement is dominated by tie-breaking, not by model quality — a perfectly calibrated model scores ≈0.75 on this sample. Second, a higher number there can be a signature of overconfidence: the RTN build's 0.915 exceeds that calibrated ceiling while its KL is worse. KL divergence, the metric the base model card reports, is the one to compare.

Additional checks: on a 28-row internal decision suite, this build and the RTN build agree on 26/28 rows (mean max probability delta 0.0135); the two deployed KV-cache precisions (bf16 vs fp8, see below) produce 28/28 identical decisions with max probability delta 0.012.

Serving with vLLM

vllm serve aioutfitters/JEV-27B-VL-AWQ \
  --served-model-name jev-awq \
  --logprobs-mode processed_logprobs \
  --max-model-len 16384 \
  --enable-prefix-caching \
  --mamba-cache-mode align \
  --limit-mm-per-prompt '{"image": 1}' \
  --max-num-seqs 8 \
  --trust-request-chat-template \
  --gpu-memory-utilization 0.95 \
  --kv-cache-dtype fp8 \
  --port 8000

System 1, client-side (the decision head's bias and the per-kind temperature from decision_head/):

import json, math, requests
from huggingface_hub import hf_hub_download

REPO = "aioutfitters/JEV-27B-VL-AWQ"
head = json.load(open(hf_hub_download(REPO, "decision_head/decision_head.json")))
temps = json.load(open(hf_hub_download(REPO, "decision_head/calibration.json")))["per_kind"]

def decide(kind, state, question, options, url="http://localhost:8000/v1/completions"):
    # kind: "choice" (options = the labels' texts), "noul" (options = ["false", "true"]),
    #       "score" (options = ["0", "1", "2", "3", "4", "5"])
    lines = [f"{chr(65 + i)}) {o}" for i, o in enumerate(options)] if kind == "choice" else list(options)
    prompt = f"[kind] {kind}\n[state] {state}\n[question] {question}\n[options]\n" + "\n".join(lines) + "\n[decision]:"
    lo, _ = head["slots"]["ranges"][kind]
    ids, bias = head["verbalizer_ids"][lo:lo + len(options)], head["bias"][lo:lo + len(options)]
    body = {"model": "jev-awq", "prompt": prompt, "max_tokens": 1, "temperature": 1.0,
            "top_p": 1.0, "top_k": 0, "add_special_tokens": False,
            "return_tokens_as_token_ids": True, "logprobs": len(ids), "logprob_token_ids": ids}
    out = requests.post(url, json=body).json()
    lp = {int(k.split(":")[1]): v for k, v in out["choices"][0]["logprobs"]["top_logprobs"][0].items()}
    z = [(lp.get(t, -1e9) + b) / temps[kind] for t, b in zip(ids, bias)]
    e = [math.exp(v - max(z)) for v in z]
    return {o: round(p / sum(e), 4) for o, p in zip(options, e)}

Memory and concurrency on one 24 GB RTX 4090 (measured, --max-model-len 16384):

The weights plus vLLM's runtime overhead (Marlin thread-tile padding, CUDA-graph pool, single-image encoder cache, the Gated DeltaNet state cache) take ~21 GiB, which is why the context window is capped at 16k here and why --kv-cache-dtype fp8 is recommended: it buys +72% KV cache at no measured latency cost and no decision change (28/28 identical). For longer contexts: raise --max-model-len with fp8 KV and/or run tensor-parallel 2 (the 4-bit weights leave ~25 GiB for KV across two cards), at the cost of a PCIe all-reduce on every prefill (~30% slower prefill in our measurements). For maximum concurrency with no communication cost, run one replica per GPU.

Notes:

• --max-num-seqs 8 is required — inherited from the base model: above 8 sequences in one batch, vLLM's LoRA path for this multimodal class returns wrong System 1 probabilities. Requests beyond 8 queue.
• When calling /v1/completions for System 1 yourself, pass top_k: 0 and top_p: 1.0 (the model's generation_config sets top_k=20, top_p=0.95, and vLLM applies them as request defaults).
• vLLM logs Marlin requires thread-tile padding for some weight shapes in this model ... performance may be degraded at startup. This is expected for the Gated DeltaNet projection shapes; the throughput numbers above already include it.
• --trust-request-chat-template is needed only for image decisions through the raw decision template.
• The vision tower is bf16 and bit-identical to the base model's.

How this was built

1. Merge the System 1 LoRA adapter (r=32, alpha=64) into the base bf16 weights — 401/401 modules, lm_head included. (Required: the decision head is an lm_head LoRA, and vLLM refuses those on quantized bases.)
2. Calibrate on the model's own corpus: 353 samples from the calibration split of SargeDev/jev-distill-corpus-v3, rendered as serving prompts (see above).
3. AWQ + quantize with llm-compressor: the smoothing mappings in recipe.yaml, then W4A16_ASYM on every Linear except lm_head and the vision tower.
4. Repair two llm-compressor 0.14 save-path behaviours: the save drops the vision tower (it loads 851 of 1184 tensors) and does not persist regex entries in quantization_config.ignore. The untouched bf16 vision tensors were spliced back from the merged checkpoint and re:.*visual.* re-added to the ignore list; without both, vLLM either misses the vision weights or demands packed 4-bit weights for 4304-dim layers.
5. Validate and benchmark with the protocol above; raw results in eval/.

Known limitations

• System 1's calibration on images is unmeasured — inherited from the base model, whose decision head was trained on text only and whose image decisions are zero-shot.
• The 16k context cap on a 24 GB card (the base model supports 256K on larger GPUs); see the serving section for options.
• Top-1 agreement with the teacher is not a good fidelity metric on flat rows — see the validation notes; use KL.
• Quantized weights change probabilities slightly by construction: expect low-thousandth differences from the bf16 model and validate any decision threshold on your own data.

License

Apache-2.0, following the base model and SargeDev/jev-distill-corpus-v3.

Original model card: autotrust/JEV-27B-VL

Everything below is the original model card of the base model, unchanged.

autotrust/JEV-27B-VL

#1 on the Decision Index Vision board

JEV-27B-VL is ranked first of 20 vision decision models on the community Jev Decision Index 0.3 — Vision board (independent evaluation, updated 7 October 2026). It is first on the Full score, first on the public benchmarks alone and first on the private tests, which are built from fresh images and never published.

• Above a 397B vision-language model: +6.4 Full-score points over the stock Qwen3.5-397B-A17B reference, with about 15× fewer total parameters.
• Zero-shot on images: the System 1 adapter and decision head were trained on text only; the vision encoder is Qwen3.8-27B's, unchanged. The board lists its stock base at 61.3 public / 51.7 private (read through a different, head-free path).
• Best on charts and documents: first on CharXiv and InfographicVQA, public and private (InfographicVQA 98.4 / 98.7), and first on the public CV-Bench and BLINK.
• Calibrated: the probabilities mean what they say (ECE 0.019 over 13,695 rows); one of four entrants below 0.02.
• About 264 ms per row on the board's RTX PRO 6000. The board ran it through this repository's own server (serve.sh: vLLM + POST /v1/decide, System 1 only).

Scores are the board's chance-corrected skill: half public vision benchmarks (CV-Bench, BLINK, RealWorldQA, CharXiv, InfographicVQA, Mind2Web, Winoground, CORD + FUNSD, Hateful Memes, R-Bench-M, MMMU-Pro vision), half private tests of the same skills.

What the board shows

1. Strong everywhere, not on one benchmark. Of the board's six skill groups, JEV-27B-VL is first in three (documents and charts, by about 4 points; spatial and perception; moderation) and second in the other three (real-world photos, web and GUI, expert reasoning). Across all 20 benchmark columns its average rank is 2.25, with 17 columns in the top three; the next models average 3.45 and 4.40.
2. It holds up on images nobody has seen. The private tests are built from fresh images and never published, so they cannot be tuned for. JEV-27B-VL is first there too (66.86).
3. A 27B model above a 397B one. +6.4 points over the stock Qwen3.5-397B-A17B with about 15× fewer total parameters; the largest gains are on questions that need a careful look at the image: BLINK private (multi-image perception) +54.0, R-Bench-M (reasoning) +30.3, CharXiv charts +17.5 public and +14.1 private.
4. The most accurate, with probabilities you can act on. 77.8% accuracy over 13,695 rows, the highest on the board, with mean confidence 79.2%: when it says 80%, it is right about 80% of the time (ECE 0.019). Thresholds on its probabilities work as expected.
5. Vision without vision training. The System 1 adapter and decision head were trained on text only and the vision encoder is Qwen3.8-27B's, unchanged: every image decision on the board is zero-shot.
6. Fast and simple to serve. About 264 ms per row and 1.41 GPU-hours for the whole board, about half the time of the runner-up, from one vLLM server (this repository's serve.sh), with no limit on the number of options.

In one sentence: the most accurate of 20 vision decision models, with no weak skill group, first on unseen images, calibrated, fast, and above a 397B model at a fifteenth of the size, from a decision head trained on text alone.

JEV-27B that can see: the same System 1 decisions and System 2, now on images

autotrust/JEV-27B-VL is autotrust/JEV-27B with vision, and the #1 model on the Jev Decision Index Vision board.

New (3 October 2026): robot arm and computer use

Every step below is one System 1 decision: a camera image or a screenshot in, a probability for every action out, in a single forward pass.

Robot arm: pick and place from a camera image. At every step System 1 looks at the top camera image and answers two questions: is the target left or right of the gripper, and above or below it? The arm moves accordingly and halves its step whenever an answer flips. It grasps the cube, carries it and drops it in the tray (MuJoCo simulation).

75% of 20 random scenes completed, the best of the JEV family. Every cube it grasped ended in the tray (15 of 15); every miss was a grasp 3.0–3.7 cm off target. About 240 ms per decision.

Computer use: screenshot → which element to click. A real browser (headless Chromium). Every clickable element gets a numbered box; System 1 picks the next click (or "the task is complete"), the browser clicks it, and the loop repeats.

95% of 60 random multi-step tasks completed (shop, settings, mail; 3–7 clicks each), about 0.26 s per click. The colour swatches carry no text, so that click is decided from the screenshot alone.

Same scenes and tasks for every model in the family:

• Use System 1 for simple visual questions inside a control loop. Asked to pick one of 8 motor commands directly, it completed 0 of 10 scenes.
• For computer use, give it the element text (as an accessibility tree would). With the numbered boxes alone it often declares the task complete too early.

Demo code: JEV-9B vl/demos/ (set JEV_URL to this server). Per-episode results: reports/demos/.

New (1 October 2026)

• System 1 over plain HTTP. serve_decide.py adds POST /v1/decide to the vLLM server: send {kind, state, question, options} and get a calibrated probability for every option, for text or images. See Quick start.
• Up to 256 options per choice question, with no retraining. Zero-shot on CLINC150 with all 150 intents as options: 89.5% with intent names alone, 93.8% with a one-line description per option. See Choice questions with up to 256 options.
• How to write prompts, measured. One-line "use when" descriptions for similar options help; question wording, JSON vs plain text and extra instructions make no measurable difference. See Writing System 1 prompts.
• 256K-token prompts. Decisions that hinge on one sentence hidden at a random depth in up to 250K tokens of text: 20 of 20 correct at every length tested. See Context length.

Highlight: zero-shot short-video recommendation (for TikTok-style feeds)

A short-video feed has to decide, for every user and every new clip, whether to show it. JEV-27B-VL System 1 can make that call from the pictures alone: it looks at the covers of the last 5 videos a user watched plus one candidate cover, and returns P(this user watches it) in one forward pass. It needs no interaction logs, no item embeddings and no training, so a brand-new video can be ranked the moment it is uploaded, before anyone has watched it.

Offline evaluation on MicroLens-100k, a public dataset from a real short-video platform with the original covers: for 200 users, the video each one actually watched next is hidden among 19 videos other users watched within ±3 days (what the feed would have been showing), and every method ranks the 20 candidates.

• Matches collaborative filtering with zero behaviour data: the same AUC as item-based CF learned from 59,045 users' watch histories (difference 0.000, 95% interval −0.041 to +0.040), and a higher top-5 hit rate (59% vs 49%).
• Solves the cold-start problem: collaborative filtering needs co-watch history; JEV only needs the cover, so new videos and new creators can be recommended from the first second.
• Pictures beat words: covers alone score +0.078 AUC over titles (95% interval +0.031 to +0.126).
• Fast enough for a ranking stage: 20 candidate covers scored in about 2.6 s on one GPU in the live demo.

Code, data pipeline and a live web demo: JEV-27B-DEMO / 07-video-recommendation.

Watch it decide

Every move below is one System 1 decision: a probability for each possible action, read off in a single forward pass. The panels show the action probabilities and the time each decision took.

Super Mario Bros. Each step is a movement choice plus a jump decision, about 150 ms per decision (video at 5× speed).

Rubik's Cube. Solving a 25-turn scramble, choosing each move from 11 standard formulas, about 46 ms per decision (10× replay).

Tetris. Choosing the column and orientation of each piece, about 122 ms per decision (5× speed).

Quick, Draw! Guessing what a player is sketching, stroke by stroke, among 16 answers. On 320 real players' sketches from the Quick, Draw! dataset it names the drawing 88% of the time once it is finished, and 62% of the time when only 60% of the strokes are down (random guessing: 6%). About 270 ms per guess.

Snake. Looking at the board and a list of positions, choosing a direction at every step: 18 pieces of food in 131 steps, about 300 ms per decision (4× playback).

Highlight: agent judge

Plan-RewardBench (ACL 2026). Which of two complete trajectories of a tool-using agent is better? On 1,171 pairs (planning, error recovery, safe refusal, tool irrelevance), System 1 reaches 73.2% macro-average accuracy, the top of the paper's table (Table 4): Qwen-Plus 70.0, DeepSeek-V3.2 69.6, Inf-ORM-Llama3.1-70B 69.2, Gemini-3-Flash 69.1, GPT-5 68.5.

AgentRewardBench (2025, with screenshots).

Did the web agent really finish the task? Given the user's goal, the agent's actions and final message, and the final screenshot, System 1 returns P(task completed) in one forward pass, zero-shot. On AgentRewardBench (McGill, 2025; 1,302 expert-labelled trajectories from GPT-4o, Claude 3.7, Llama 3.3 and Qwen2.5-VL agents) its precision is higher than every judge on the official leaderboard at that judge's own recall:

At threshold 0.5: precision 78.4, recall 70.2; AUROC 0.91. All 1,302 judgements in about 4 minutes on one GPU. Code: JEV-27B-DEMO / 10-agent-judge.

Highlight: multimodal judge

Given an image, a question and two answers, System 1 says which answer is better, zero-shot and in one forward pass per order. On VL-RewardBench (CVPR 2025, 1,247 human-verified pairs) it reaches 78.3% overall accuracy, above every model on its official leaderboard (26 models, updated May 2025):

2,494 decisions in 163 seconds on one GPU. Code: JEV-27B-DEMO / 09-multimodal-judge.

On the newer Multimodal RewardBench 2 (Meta, December 2025), which includes the current generation of judges (1,000 expert-annotated pairs per task; other rows from the paper's Table 2):

Text-to-image judging comes within 1.3 points of GPT-5; the four-task average is at the level of GPT-4.1 and Qwen3-VL-32B. GPT-5 and Gemini 3 Pro stay ahead, and image editing is the weakest task.

Benchmarks

JEV-27B-VL gives the same text decisions as autotrust/JEV-27B: on 1,000 answer-checking decisions the mean probability difference is 0.010 and 99.8% fall on the same side of 0.5. The text results below were measured with JEV-27B; the image result was measured with JEV-27B-VL.

Six public decision benchmarks

Scores in %, higher is better. JEV-27B and TypeSafe Jev 1.13 (hosted API) were run in full by AutoTrust (27 September 2026); the other rows are as reported in the NeoHorse-Jev-4B evaluation.

System 1 fidelity and calibration

Held-out test_set_30k of jev-distill-corpus-v3.

Independent benchmark: decision-models-under-pressure

Human gold labels (CLINC-150, MTOP, GoEmotions, DBpedia), 800 items.

Applied tasks (from JEV-27B-DEMO)

System 2

Quick start

hf download autotrust/JEV-27B-VL --local-dir JEV-27B-VL
bash JEV-27B-VL/serve.sh          # vLLM on :8000; one GPU with 80 GB or more

serve.sh runs serve_decide.py: the standard vLLM OpenAI server, with the same flags as vllm serve, plus a POST /v1/decide route for System 1.

python3 JEV-27B-VL/serve_decide.py --model JEV-27B-VL --served-model-name autotrust/JEV-27B-VL \
  --enable-lora --max-lora-rank 32 --lora-modules jev-decision=JEV-27B-VL/adapter_vllm \
  --logprobs-mode processed_logprobs --max-model-len 32768 --enable-prefix-caching --mamba-cache-mode align \
  --limit-mm-per-prompt '{"image": 8}' --max-num-seqs 8 --trust-request-chat-template

Set MAX_MODEL_LEN=262144 before serve.sh for the full 256K context (see Context length).

System 1: POST /v1/decide

Send a question and get back a calibrated probability for every option. The server builds the decision prompt, reads the option tokens and applies the decision head's bias and temperature, so any HTTP client works.

curl localhost:8000/v1/decide -H 'Content-Type: application/json' -d '{
  "kind": "choice",
  "state": "Customer: my card was charged twice for one coffee.",
  "question": "Which team should handle this?",
  "options": ["billing", "shipping", "tech support"]}'

{"kind": "choice", "effective_kind": "choice", "options": ["billing", "shipping", "tech support"],
 "probabilities": [0.9969, 0.0000, 0.0031], "choice_index": 0, "choice": "billing",
 "adaptation": "native", "protocol": "jev27-bare-v1", "model": "autotrust/JEV-27B-VL",
 "usage": {"prompt_tokens": 49, "completion_tokens": 1, "total_tokens": 50}, "num_model_requests": 1, "elapsed_seconds": ...}

For images, make state a list; each image is placed where it appears. Images can be data URLs or https:// URLs.

import base64, requests

def image(path):
    return {"image": "data:image/jpeg;base64," + base64.b64encode(open(path, "rb").read()).decode()}

def decide(kind, state, question, options=None):
    body = {"kind": kind, "state": state, "question": question, **({"options": options} if options else {})}
    r = requests.post("http://localhost:8000/v1/decide", json=body).json()
    return dict(zip(r["options"], r["probabilities"]))

decide("noul", ["Photo: ", image("photo.jpg")], "Is this scenario one where: the image shows food or cooking?")
# e.g. {'false': 0.00, 'true': 1.00}
decide("choice", ["Listing photo: ", image("item.jpg"), "\nSeller title: wireless earbuds, barely used"],
       "Which category fits this listing?", ["electronics", "clothing", "home and kitchen", "toys"])

Text-only decisions give the same results as autotrust/JEV-27B. GET /v1/decide/info reports the option limit and the temperatures.

import json, math, requests

B = "JEV-27B-VL"
DH = json.load(open(f"{B}/adapter_vllm/decision_head.json"))
T = json.load(open(f"{B}/calibration.json"))["per_kind"]
RAW = ("{%- for m in messages -%}{%- for c in m['content'] -%}{%- if c['type'] == 'text' -%}{{ c['text'] }}"
       "{%- else -%}<|vision_start|><|image_pad|><|vision_end|>{%- endif -%}{%- endfor -%}{%- endfor -%}")

def decide_mm(kind, state_parts, question, options=None):
    """state_parts: list of str and {'image_url': {'url': ...}, 'type': 'image_url'} parts. Up to 16 options."""
    options = {"noul": ["false", "true"], "score": [str(i) for i in range(6)]}.get(kind, options)
    lines = options if kind != "choice" else [f"{'ABCDEFGHIJKLMNOP'[i]}) {o}" for i, o in enumerate(options)]
    content = [{"type": "text", "text": f"[kind] {kind}\n[state] "}]
    content += [p if isinstance(p, dict) else {"type": "text", "text": p} for p in state_parts]
    content.append({"type": "text", "text": f"\n[question] {question}\n[options]\n" + "\n".join(lines) + "\n[decision]:"})
    s = DH["slots"]["ranges"][kind][0]
    ids = DH["verbalizer_ids"][s: s + len(options)]
    r = requests.post("http://localhost:8000/v1/chat/completions", json={
        "model": "jev-decision", "messages": [{"role": "user", "content": content}], "chat_template": RAW,
        "add_generation_prompt": False, "add_special_tokens": False, "max_tokens": 1, "temperature": 1.0,
        "top_k": 0, "top_p": 1.0, "logprobs": True, "top_logprobs": len(options), "allowed_token_ids": ids,
        "return_tokens_as_token_ids": True}).json()
    lp = {int(t["token"].split(":")[1]): t["logprob"] for t in r["choices"][0]["logprobs"]["content"][0]["top_logprobs"]}
    z = [(lp.get(t, -1e9) + DH["bias"][s + i]) / T[kind] for i, t in enumerate(ids)]
    e = [math.exp(x - max(z)) for x in z]
    return {o: x / sum(e) for o, x in zip(options, e)}

System 2 on images

chart = {"type": "image_url", "image_url": {"url": image("chart.png")["image"]}}   # OpenAI-style image part
requests.post("http://localhost:8000/v1/chat/completions", json={
    "model": "autotrust/JEV-27B-VL",
    "messages": [{"role": "user", "content": [chart, {"type": "text", "text": "What does this chart show?"}]}],
    "max_tokens": 1024, "chat_template_kwargs": {"enable_thinking": False}})

Choice questions with up to 256 options

choice takes 2–256 options in one question. The first 16 are labelled A–P, the labels the decision head was trained on. Beyond 16, the labels continue with Q–Z and then two-letter labels that are single tokens (AA, AB, …), read the same way, with no retraining. The response's adaptation field says native (up to 16 options) or wide-labels.

Zero-shot intent classification with every intent offered at once (accuracy on 400 test utterances per row):

• One choice question beats one yes/no question per option, and costs one forward pass instead of one per option: 83.2% vs 78.0% on MASSIVE, 78.8% vs 67.0% on BANKING77 (64 options), 94.5% vs 82.0% on CLINC150 (64 options).
• Calibration at 150 options (CLINC150): ECE 0.079 with names only (mean confidence 0.84 against 89.5% accuracy, slightly under-confident) and 0.034 with descriptions.
• Speed (one B200, 8 requests in flight, --max-num-seqs 8): a 150-option question takes a median 1.4 s with names only (about 790 prompt tokens) and 2.6 s with descriptions (about 3,100 tokens). Above 128 options the server reads the option tokens in two passes; the second reuses the cached prompt.

Writing System 1 prompts

What we measured on the intent-classification sets above (400 utterances per variant):

1. Facts in state, one question in question. state holds everything the decision depends on; question asks one thing about it. Plain text and JSON both work: a JSON state scored the same as plain text (83.8% vs 83.2%, 79.0% vs 79.0%, 94.5% vs 94.8%).
2. Ask about all candidates in one choice question, not one yes/no question per candidate (see above).
3. Give similar options a one-line "use when" description. This was the only change that clearly helped: +4.8 points on BANKING77 (64 options), +2.5 on CLINC150 (64) and +4.3 on CLINC150 (all 150). The line should say what separates the option from its neighbours: [options] A) card arrival: use when users ask about the status or delivery of a physical card. B) card delivery estimate: use when users ask about the expected arrival time or delivery speed of a new card. C) Refund not showing up: use when a customer cannot see a completed refund in their account or statement. D) request refund: use when a user asks how to initiate or the timing of a refund for a purchase. More text did not help further. Longer rules with "do not use for …" clauses (about three times the tokens) and two worked examples per option scored 83.8% and 84.0% on BANKING77 (vs 83.8% for one line), and 97.3% and 96.0% on CLINC150 (vs 97.3%). Generic descriptions did not help either: CLINC150's own intent descriptions scored 94.5%, against 94.8% for names alone.
4. Wording barely matters. A domain-specific question, an extra instruction ("Choose the single best-matching intent"), a JSON state and alphabetical option order all stayed within ±1.5 points of the plain prompt, inside the ±1.8-point sampling noise of 400 items. Keep prompts plain.
5. Phrase noul so that "true" is the outcome you want the probability of, e.g. "Is the customer asking for a refund?". Our demos use "Is this scenario one where: …?" for properties of a text or an image.
6. One option per line. The template lists one option per line, so keep newlines out of option text.
7. Check order sensitivity when it matters. About 7% of 16-option answers change with option order alone. For pairwise judging, ask both orders and average; on RewardBench pairs the two orders agreed 96% of the time.
8. Use the probability, not just the top option. Act automatically above a threshold you validated on your own data, and send the rest to System 2 or a person.

Context length

The backbone's native context is 262,144 tokens (256K), and System 1 accepts prompts up to that length when vLLM is started with --max-model-len 262144. We tested decisions that hinge on a single sentence placed at a random depth in long real text (concatenated PubMedQA abstracts): a yes/no question (was the refund for order #N approved or rejected?) and a 16-option question (which room is the spare part for machine M kept in?), 10 of each per length.

Latency is one request at a time on one B200 and is almost all prompt processing (about 8,700 tokens per second at 250K).

Memory: the KV cache takes about 65 KB per token, so a full 256K prompt needs about 17 GB on top of the 52 GB of weights (tested on a 183 GB B200). On an 80 GB GPU, start with --max-model-len 131072.

Serving notes

• --max-num-seqs 8 is required. With more than 8 sequences in one batch, vLLM's LoRA path for this multimodal model class returns wrong System 1 probabilities (the text-only JEV-27B is not affected). With the cap, results match JEV-27B at any client concurrency; requests beyond 8 simply queue.
• Throughput on one B200 with the cap: 24 short text decisions in about 1.4 s; 4,000 image decisions with 6 images each in 316 s (about 13 per second).
• --trust-request-chat-template lets /v1/decide (and the client-side code above) render the raw decision template for image decisions. System 2 uses the model's own chat template.
• serve_decide.py needs the vLLM development build this model was tested with (September 2026): it uses logprob_token_ids and that build's server layout. Everything else is plain vLLM, and all OpenAI endpoints stay available on the same port.
• Calling /v1/completions or /v1/chat/completions for System 1 yourself: pass top_k: 0 and top_p: 1.0. The model's generation config sets top_k=20 and top_p=0.95; vLLM applies them as request defaults, and they would truncate the returned probabilities. serve_decide.py does this for you.
• Images are resized by the Qwen3.8 processor. Downscaling large images first (for example to at most 448 px) keeps the number of vision tokens and the latency low.

Limitations

• System 1's decision head was trained on text. Decisions over images are zero-shot: they are well ordered in the experiment above, but their calibration on image tasks has not been measured systematically.
• The short-video recommendation result is an offline evaluation on one dataset (200 users), ranking from covers only.
• Choices with more than 16 options use labels the decision head never saw in training. Accuracy holds up (see the intent-classification table), but their calibration has been measured on one dataset only.
• The long-context test is a single-fact decision over text. Long documents with images and harder long-document reasoning have not been measured.

License

Apache-2.0. This repository contains the weights of Qwen/Qwen3.8-27B (Apache-2.0, see LICENSE) unchanged, plus the JEV System 1 adapter and decision head from autotrust/JEV-27B.