A LoRA on Qwen/Qwen3.8-27B for typed decisions over text and images. It continues from
e12ex2/jev-vision-27b and adds training on synthetic decision
questions whose answers are known by construction.
Give it a state (text, images, or both), a question and a list of options; it returns a probability for every option
from one forward pass. Nothing is generated.
Results
Decision Index 0.3, public vision suite (12,210 rows, all answered), official kit:
| Benchmark |
jev-vision-27b (base) |
Herbert v0.5 |
| CV-Bench |
76.8 |
79.2 |
| BLINK |
54.6 |
72.8 |
| RealWorldQA |
72.8 |
73.4 |
| CharXiv |
78.9 |
77.4 |
| InfographicVQA |
96.2 |
96.7 |
| Mind2Web |
84.4 |
83.1 |
| Winoground |
83.5 |
84.0 |
| KIE (CORD + FUNSD) |
91.3 |
92.5 |
| Hateful Memes |
52.0 |
52.7 |
| R-Bench-M |
24.7 |
31.2 |
| MMMU-Pro vision |
38.0 |
42.3 |
| Public vision index |
73.06 |
75.69 |
Run files: e12ex2/herbert-results.
How it answers
Options are labelled A, B, C, ... in the prompt; the answer is read from the label logits at the answer position
and softmaxed over the listed options only. Up to 624 options per question (A-Z, a-z, 0-9, then single-token
two-letter labels); larger questions are refused as unsupported, never truncated. Images up to 8 per request.
Run it
pip install vllm fastapi uvicorn peft transformers "decision-index[vision] @ git+https://github.com/apolinario/decision-index"
cd code
python -m train.merge --ckpt .. --out ../merged # adapter -> plain bf16 model (one-time)
python -m serve.server --model ../merged --perms 1 --port 8000 --max-model-len 65536 --max-pixels 1638400
POST /v1/systemone (Decision Index wire format) or /v1/decisions (the kit's vision decide engine, images as a
separate list).
Training
- Starts from the jev-vision-27b adapter (LoRA r=32, alpha=64 on the language model; vision tower frozen).
- Added data: synthetic decision questions generated by our own renderer (3D scenes with depth, point
correspondence across views, camera motion, spatial relations and counts; charts drawn from known data; exam-style
pages with computed answers), labels exact by construction. No benchmark items and no evaluation splits were used.
- The previous mix (text decision corpus plus train-split vision data, see the base model's card) is replayed
alongside it.
Limits
Public-suite numbers are self-run with the official kit; the board's Full score also includes private tests run by the
maintainers.
