OddEye 27B (experimental)
One street photo in, a structured Japanese observation record out. OddEye is the street-reading model of michiyomi. It is built on Qwen3.8-27B and compressed to 4-bit (NVFP4, about 20 GB), so a single GPU — even an RTX 5090 — can run it.
It keeps what it sees (observations) and what it infers (interpretations, each with a confidence label) in separate fields. The record has 13 top-level fields, including objects, text on signs, relations, surfaces, standard road facets (lanes, sidewalks, lighting…), interpretations and a scene summary.
Results
- Overconfident statements and misread text: no significant difference from the BF16 original, and clearly fewer than a generic 4-bit version.
- Speed: 1.8× the original on an RTX PRO 6000. On an RTX 5090, about 27 s per photo, or about 900 photos/h with 8 requests at once.
- Weak points: more generic interpretations than the original; some text is still misread.
Evaluation details
All numbers use the recommended prompt (v5.2c), greedy decoding and the same photos for every model. "Generic NVFP4" is a publicly distributed 4-bit version of the same base model; "GPTQ only" is our own 4-bit calibration on michiyomi requests, without training. ↑ higher is better, ↓ lower is better.
Blind comparison (60 street photos)
|
BF16 original |
Generic NVFP4 |
GPTQ only |
OddEye |
| Reference-fact score ↑ |
0.577 |
0.561 |
0.569 |
0.574 |
| Contradicted reference facts ↓ |
8.1% |
10.4% |
9.9% |
9.1% |
| Overconfident statements / photo ↓ |
0.57 |
0.85 |
0.82 |
0.55 |
| Clear errors / photo ↓ |
0.37 |
0.42 |
0.30 |
0.25 |
| — misread text ↓ |
0.05 |
0.25 |
0.13 |
0.10 |
| — objects that are not there ↓ |
0.17 |
0.08 |
0.05 |
0.07 |
| Useful observations beyond the references / photo ↑ |
1.27 |
1.43 |
1.53 |
1.40 |
| Interpretations: useful / generic |
1.18 / 0.53 |
1.20 / 0.55 |
1.28 / 0.78 |
1.18 / 0.88 |
Differences, with 95% confidence intervals (paired bootstrap over photos):
| OddEye minus … |
Generic NVFP4 |
BF16 original |
| Overconfident statements / photo |
−0.30 [−0.47, −0.13] |
−0.02 [−0.17, 0.13] |
| Misread text / photo |
−0.15 [−0.28, −0.03] |
+0.05 [−0.02, 0.15] |
| Clear errors / photo |
−0.17 [−0.33, 0.02] |
−0.12 [−0.32, 0.08] |
| Reference-fact score |
+0.013 [−0.011, 0.037] |
−0.003 [−0.027, 0.019] |
| Generic interpretations / photo |
+0.33 [0.12, 0.53] |
+0.35 [0.17, 0.53] |
Two later blind rounds on the same photos (each again with the BF16 original) found no significant difference from BF16 on the error measures either. In all three rounds OddEye misread slightly more text than BF16 (+0.03 to +0.05 per photo; the 95% intervals reach zero).
By kind of fact
Score and share of contradicted facts, per kind of reference fact (867 facts in total).
| Kind of fact (facts) |
BF16 original |
Generic NVFP4 |
OddEye |
| Text on signs (104) |
0.69 · 13% |
0.58 · 24% |
0.64 · 16% |
| Relations between objects (170) |
0.48 · 6% |
0.47 · 8% |
0.51 · 4% |
| Counts (97) |
0.55 · 9% |
0.54 · 10% |
0.54 · 8% |
| Appearance (148) |
0.55 · 4% |
0.52 · 6% |
0.55 · 5% |
| Basic layout (146) |
0.63 · 16% |
0.62 · 16% |
0.62 · 19% |
| Transient things: vehicles, people, works (114) |
0.58 · 7% |
0.59 · 7% |
0.56 · 11% |
| Context (88) |
0.59 · 1% |
0.64 · 3% |
0.61 · 2% |
Reading text is where generic 4-bit quantisation hurts most, and where OddEye recovers most (score +0.075 [0.011, 0.153] over Generic NVFP4). Fewer relation contradictions than Generic NVFP4 (−4.3 points [−8.3, −0.6]). Transient things are a weak spot: 2.8 points [0.6, 5.6] more contradictions than BF16.
Agreement with the original model (140 photos)
Teacher-forced on the BF16 original's own answers: how much less likely each 4-bit model finds them.
|
Generic NVFP4 |
GPTQ only |
OddEye |
| Extra negative log-likelihood per token ↓ |
0.051 |
0.040 |
0.015 |
| Same top-1 token as BF16 ↑ |
95.5% |
95.9% |
95.9% |
Output structure (140 photos)
|
BF16 original |
Generic NVFP4 |
OddEye |
| All 13 top-level fields present ↑ |
140 |
140 |
138 |
| Stopped at the 8,192-token limit ↓ |
0 |
0 |
2 |
| Median output tokens |
3,151 |
3,383 |
3,339 |
Speed and size
|
BF16 original |
Generic NVFP4 |
OddEye |
| Weights |
55.6 GB |
20.6 GB |
20.6 GB |
| RTX PRO 6000 (96 GB), 32 concurrent, images/h ↑ |
1,020 (with MTP) |
1,749 |
1,833 |
| RTX 5090 (32 GB) |
does not fit |
not tested |
27.5 s per photo; ~900 images/h with 8 concurrent |
Variants we tried (not released)
| Variant |
Result against OddEye |
| Same training, 1,000 answers instead of 400 |
Closer to BF16 token by token, but more overconfident: +0.22 statements / photo [0.07, 0.37] |
| 1,000 answers with KL distillation (BF16 top-20 distribution) |
More contradicted facts (+2.3 points [0.3, 4.4]) and slower (1,659 images/h) |
| Answers re-written as one-line JSON (37% fewer tokens) |
Broken nesting in 103 of 140 outputs; 31 ran to the token limit |
How it was evaluated
- 60 photos: 20 from a fixed panel plus 40 from the evaluation pool, about 14.5 reference facts each. Structure, speed and agreement use 140 photos. All evaluation photos come from image sequences not used for training.
- A Claude model (Anthropic) graded each photo with all outputs side by side, without knowing which model wrote which, using a fixed rubric (facts held / partial / missed / contradicted, overconfident statements, clear errors, useful extras, interpretation quality). The graders looked at the photo, zooming in on small text.
- Limits: the references were written by AI and not checked by people (one known error); the prompt revisions v5.2b/v5.2c were made from errors on these same photos, so absolute values are optimistic. Read the numbers as a comparison between models.
Quick start
vllm serve finalvent/oddeye-27b-nvfp4 --served-model-name oddeye \
--max-model-len 16384 --kv-cache-dtype fp8 --mamba-ssm-cache-dtype bfloat16 \
--limit-mm-per-prompt '{"image":1}' \
--speculative-config '{"method":"mtp","num_speculative_tokens":2}' --enable-prefix-caching
# RTX 5090 (32 GB): add --max-num-seqs 8 --gpu-memory-utilization 0.92
python3 example_request.py photo.jpg > observation.json
Use the included prompt, prompt_v5.2c.md; the model was specialised on it. example_request.py (standard library only) sends exactly the request that was evaluated. Tested with vLLM 0.27–0.28 on Blackwell GPUs.
Before you use the output
- It describes only what is visible in that photo, at the time it was taken — not the current state of the place.
- Widths and lane counts are estimates from one image, not measurements. "Not visible" is a real answer; don't fill in what the record does not say.
- It occasionally transcribes a licence-plate number (1 photo in 140, as did the original). Redact plates and personal names before publishing outputs.
- Don't use it as the only basis for safety or accessibility decisions.
About
- Experimental release from the michiyomi project, for the Tokyo Governor's Cup 2026 Final Stage. A full version will follow on the next Qwen generation.
- Trained only on the base model's own answers (4-bit-aware self-distillation on 400 photos); no outputs of external AI services were used. Evaluation: blind comparison on 60 photos, graded by Claude (Anthropic) against AI-written references — read the numbers as a comparison between models.
- Training photos: street-level imagery from Mapillary (not included).
- License: Apache-2.0 (same as the base model). No conditions on the outputs. The example figure contains a photo by nashitakahashi via Mapillary and is shared under CC BY-SA 4.0.
日本語
街路の写真1枚から、日本語の構造化された観測記録（JSON）を作る、みちよみの実験版モデルです。Qwen3.8-27Bを元に、約20GBの4bit（NVFP4）に縮めて特化させました。RTX 5090を1枚積んだ機材でも動きます。
- 見えたもの（観測）と推し量ったこと（解釈）を分けて書きます。解釈には確度が付きます。
- 断定のしすぎと文字の誤読は、元のモデルと有意な差がありません。一般的な4bit版より少なくなりました。速さは元のモデルの約1.8倍です（RTX PRO 6000）。RTX 5090では1枚あたり約27秒です。
- 評価の詳細（60枚の盲検比較、事実の種類ごとの得点、元のモデルとの一致度、出力の構造、速さ、試した別の作り方）は、英語の「Evaluation details」の表にあります。文字の読み取りは、一般的な4bit版より明らかに良くなりました（得点+0.075）。走っている車など一時的なものは、元のモデルより矛盾がやや多い弱点です。
- 使い方は上のQuick startのとおりです。同梱の推奨プロンプト
prompt_v5.2c.md を使ってください。
- 出力は、撮影時点の写真に見えたものだけを書いたものです。幅などは推定値です。まれにナンバーを書き写すことがあるので、公開するときはナンバーや個人名を伏せてください。
- 重みはApache-2.0です。出力には条件を付けません。例の図には、Mapillaryのnashitakahashiさんの写真（CC BY-SA 4.0）を使っています。
