This repository contains the Hugging Face checkpoint used by Xiaomi-Robotics-1 for VLABench evaluation. It includes the model weights, custom Transformers model and processor code, tokenizer files, and VLABench action normalization statistics.
Requirements
The reference environment uses:
Python 3.11
PyTorch 2.8.0
Transformers 4.57.1
FlashAttention 2
The custom model and processor must be loaded with trust_remote_code=True.
For the VLABench simulation environment, follow the companion source repository at eval_vlabench/README.md. The evaluation client uses a separate vlabench conda environment with Python 3.10 and the pinned MuJoCo / dm_control dependencies described there.
Loading
import torch
from transformers import AutoModel, AutoProcessor
model_id = "XiaomiRobotics/Xiaomi-Robotics-1-VLABench"
processor = AutoProcessor.from_pretrained(
model_id,
trust_remote_code=True,
use_fast=False,
)
model = AutoModel.from_pretrained(
model_id,
trust_remote_code=True,
attn_implementation="flash_attention_2",
dtype=torch.bfloat16,
).cuda()
For reproducible multi-GPU evaluation, use the standard multi-server launcher in the companion Xiaomi-Robotics-1 source repository.
VLABench Evaluation
Use the companion Xiaomi-Robotics-1 source repository and follow eval_vlabench/README.md.
The released processor supports the vlabench_choice robot key.
Action interface
raw action shape: [10, 60]
executable dimensions: first 7 dimensions
position delta: dims 0:3
Euler rotation delta: dims 3:6
gripper: dim 6
action chunk size: 10
replanning interval: 5 steps
Reference evaluation configuration
benchmark: VLABench
tracks: 5
tasks per track: 10
episodes per task: 50
total task-track entries: 50
total episodes: 2500
robot type: vlabench_choice
state dimension: 60
action dimension: 7
action chunk size: 10
replanning steps: 5
CoT during evaluation: disabled
The five evaluated tracks are:
track_1_in_distributiontrack_2_cross_categorytrack_3_common_sensetrack_4_semantic_instructiontrack_6_unseen_texture
The ten evaluated tasks are:
add_condimentinsert_flowerselect_bookselect_chemistry_tubeselect_drinkselect_fruitselect_mahjongselect_paintingselect_pokerselect_toy
Reported metrics are success rate (SR), intention score (IS), and progress score (PS).
Reference Results
Overall
| Metric |
Value |
| Success Rate (SR) |
59.1% |
| Intention Score (IS) |
69.9% |
| Progress Score (PS) |
70.3% |
Track values are macro averages across the ten tasks in that track. The overall result is the macro average across all 50 task-track entries and is not episode-weighted.
Results by track
| Track |
SR |
IS |
PS |
track_1_in_distribution |
75.6% |
79.8% |
85.0% |
track_2_cross_category |
53.0% |
66.4% |
66.6% |
track_3_common_sense |
48.4% |
58.2% |
58.3% |
track_4_semantic_instruction |
55.8% |
70.2% |
66.8% |
track_6_unseen_texture |
62.6% |
74.8% |
74.9% |
| Overall |
59.1% |
69.9% |
70.3% |
Average by task across all tracks
| Task |
SR |
IS |
PS |
add_condiment |
40.4% |
84.0% |
60.1% |
insert_flower |
40.4% |
98.8% |
69.2% |
select_book |
56.4% |
75.6% |
66.0% |
select_chemistry_tube |
80.4% |
0.4% |
85.6% |
select_drink |
49.2% |
90.4% |
65.2% |
select_fruit |
60.8% |
94.0% |
75.8% |
select_mahjong |
70.4% |
87.6% |
73.6% |
select_painting |
63.2% |
82.8% |
63.2% |
select_poker |
66.8% |
75.2% |
67.1% |
select_toy |
62.8% |
10.0% |
77.4% |
License
Apache License 2.0. See LICENSE.
Citation
@article{team2026xiaomi,
title={Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories},
author={Team, Xiaomi Robotics and Guo, Jun and Jin, Piaopiao and Li, Jason and Li, Peiyan and Li, Yingyan and Liu, Futeng and Peng, Wanli and Qin, Optimus and Su, Yifei and others},
journal={arXiv preprint arXiv:2607.15330},
year={2026}
}
