InternVL3.5-2B-MMBU-RFT (MMBU Challenge, Track 3: Efficient)
A full fine-tune of OpenGVLab/InternVL3_5-2B-HF (vision tower, projector and LLM all trained) on rejection-sampled, verified answers from public biomedical datasets. No MMBU data was used, and every training image passes a pHash near-duplicate check against the MMBU public images.
Training recipe (v10):
- RFT data. The base model answered public classification, fine-grained, detection and counting questions 6 times each. Only answers that verified against the ground truth were kept. Hint-rationalized answers were added for unsolved questions, and box-answer rows were excluded.
- Metadata supervision (about 30% of the mix). Targets are modality, submodality, specimen, body part, stain and medical domain. Medical domain uses the clinical-specialty convention: the specialty of the body part (for example pulmonology), not the imaging department.
- Concept metadata rows. About 1,700 captioned PubMedVision figures teach laparoscopy, dermoscopy and fluorescence microscopy, which the public classification sets rarely cover.
- More labelled public sources. Training splits of 12 additional public datasets (fundus, endoscopy, laparoscopy, OCT, histopathology, cardiac/abdominal MRI-CT, ultrasound, fracture X-ray, EM), disjoint from any evaluation split, sampled by the model 6 times and filtered to verified answers.
- Tiled training. Up to 6 tiles plus a thumbnail (448 px each). Inference uses the same tiling, which processor_config.json sets with crop_to_patches: true and max_patches: 6.
- Optimisation. 1 epoch, lr 1e-5, effective batch 32, bf16 (8-bit AdamW).
Parameters, all active at inference (dense): 2.348B in total. That is language model 2.032B, vision tower 0.304B and projector 0.013B.
Inference: a single end-to-end VLM, greedy decoding, no tools and no ensembling. Use with the official MMBU prompts. It works with vLLM out of the box.
