InternVL3-8B CoT SFT (vision-unfrozen LoRA, merged) — checkpoint 1100
A multi-image chain-of-thought fine-tune of
OpenGVLab/InternVL3-8B, merged into standalone
weights so it loads with trust_remote_code=True in transformers or vLLM without the adapter.
This is a mid-training checkpoint: step 1100, epoch 0.35 of an unfinished run. It is not a
final model.
Training
- LoRA, r=64, alpha=128, dropout=0.05, on
q/k/v/o_proj and gate/up/down_proj
modules_to_save: ["vision_model", "mlp1"] — the vision tower and the projector were trained
in full rather than low-rank adapted, which is why the adapter was 1.3 GB and why merging is
required for serving- bf16, lr 1e-5
- Validation set: TIGER-Lab/Mantis-Eval
Evaluation
Accuracy against the base model. The prompt is stated per row, because it matters more than the
fine-tune does: rows are only comparable to other rows evaluated the same way.
| Benchmark |
Prompt |
InternVL3-8B |
This checkpoint |
| Mantis-Eval |
standard, no CoT |
66.36 |
67.28 |
| MIRB |
standard, no CoT |
58.93 |
56.86 |
| MuirBench |
CoT + <answer> extraction |
— |
53.15 |
| BLINK |
CoT + <answer> extraction |
56.50 |
55.18 |
MIRB and MuirBench/BLINK use different image tiling: the CoT runs cap
max_dynamic_patch at 4 so that many-image questions fit the context window, while the
no-CoT rows use the default of 12. The config shipped here carries the default 12.
A note on the training metric: eval/accuracy_mcq during training is teacher-forced per-token
accuracy on Mantis-Eval, not a benchmark score, and is not comparable to any number above.
Notes on the weights
Merging was done with peft==0.20.0 under transformers==4.57; InternVL3's remote code defines
_tied_weights_keys and does not load under transformers>=5.0, which expects
all_tied_weights_keys.
