Search public pages, research tools, and SAVRN solutions.
EgoSteer-3B-Base is an open-weight model for robotics from EgoSteer, released under Apache License 2.0. Its published files total 5.6 GB. It draws 24 downloads a month.
EgoSteer-3B-Base is the base world-model-enhanced Vision-Language-Action (VLA) policy from EgoSteer: A Full-Stack System Towards Steerable Dexterous Manipulation from Egocentric Videos.
By EgoSteer, published under apache-2.0, revision 3e299fb4e2ab.
EgoSteer-3B-Base is the base world-model-enhanced Vision-Language-Action (VLA) policy from EgoSteer: A Full-Stack System Towards Steerable Dexterous Manipulation from Egocentric Videos. It is built on a Qwen3-VL backbone with a flow-matching action expert and a DINOv3 latent future-prediction objective used only during training, and learns a unified human-to-robot action space from large-scale egocentric human videos. This is the pretrained base checkpoint, corresponding to EgoSteer-9.6K-PT in the paper. For a robot-ready generalist, see Our full-stack system integrates EgoSmith (data pipeline), Robot Stack (deployment), and EgoSteer (policy) to learn from 9.6k hours of large-scale…
EgoSteer-3B-Base is the base world-model-enhanced Vision-Language-Action (VLA) policy from EgoSteer: A Full-Stack System Towards Steerable Dexterous Manipulation from Egocentric Videos. It is built on a Qwen3-VL backbone with a flow-matching action expert and a DINOv3 latent future-prediction objective used only during training, and learns a unified human-to-robot action space from large-scale egocentric human videos.
This is the pretrained base checkpoint, corresponding to EgoSteer-9.6K-PT in the paper. For a robot-ready generalist, see EgoSteer-3B-RealMan.
Our full-stack system integrates EgoSmith (data pipeline), Robot Stack (deployment), and EgoSteer (policy) to learn from 9.6k hours of large-scale egocentric human videos and facilitate data-efficient real-robot post-training, enabling steerable dexterous manipulation across over 40 tasks alongside few-shot adaptation to complex, long-horizon tasks. EgoSteer-3B-Base is the pretrained base and a starting point for data-efficient post-training on real robots.
| Component | Description |
|---|---|
| Backbone | Qwen3-VL-2B-Instruct |
| Action expert | Flow-matching (DiT / AdaLN) expert reusing the backbone KV prefix |
| World model expert | Regresses future-frame features from a frozen DINOv3 ViT-L/16 teacher, training only |
| Action space | Unified human-to-robot space based on wrist poses and fingertip keypoints |
| Total parameters | ~3B |
The policy maps a language instruction plus RGB and proprioception to a chunk of future actions.
Keep these consistent with the bundled config.yaml and normalizer.pkl.
| Specification | |
|---|---|
| Instruction | A natural-language task description |
| Cameras | Single RGB — head, 384 × 384, 6-frame history (stride 30 over a 30 fps base) |
| Intrinsics | Per-camera [fx, fy, cx, cy] (head) — required. By default rendered into the VLM prompt as text (camera_intrinsic_mode: text); rescaled to match the resized image |
| Proprioception (state) | 48-D: bimanual wrist poses (2 × [3 translation + 6D rotation] = 18), expressed in the camera frame + fingertip keypoints (2 hands × 5 fingertips × 3D = 30), expressed in wrist frame, 6-frame history |
| Action output | 48-D relative action in the same layout as the state, and relative to the current state, predicted as a 32-step action chunk |
| Normalization | State/action normalized with the bundled normalizer.pkl (relative action space) — required for inference and fine-tuning |
| Model | Parameters | Description |
|---|---|---|
| EgoSteer-3B-Base (this repo) | 3B | Base EgoSteer model trained on 9.6k hours of egocentric human videos, ready for fine-tuning |
| EgoSteer-3B-RealMan | 3B | Generalist post-trained on real-world data collected on the RealMan robot |
| File | Description |
|---|---|
model_bf16.pt |
bf16 model weights |
config.yaml |
Pretraining config, for rebuilding the network at eval and inference. Fine-tuning uses your own config, weights only |
normalizer-relative-10k-pretrain/normalizer.pkl |
State/action normalizer calculated on 9.6k hours of egocentric human videos in relative action space |
This is a custom policy, loaded and run with the
EgoSteer codebase using the bundled
config.yaml to rebuild the network.
EgoSteer depends on two pretrained backbones; download them ahead of time:
EgoSteer-3B-Base is pretrained on 9.6k hours of egocentric human videos, framed in a unified human-to-robot action space (wrist poses + fingertip keypoints). The EgoSteer-3B-RealMan variant is additionally post-trained on real-world demonstrations collected on the RealMan robot. See the paper and project page for details.
@misc{zhong2026egosteerfullstacksteerabledexterous,
title={EgoSteer: A Full-Stack System Towards Steerable Dexterous Manipulation from Egocentric Videos},
author={Yifan Zhong and Zhang Chen and Tianrui Guan and Fanlian Zeng and Yuyao Ye and Tianjia He and Ka Nam Lui and Jiayi Li and Tingrui Zhang and Ruilin Yan and Xinhao Ji and Guangyu Zhao and Wenjie Lou and Jiayuan Zhang and Yuanpei Chen and Yaodong Yang},
year={2026},
eprint={2607.09701},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2607.09701},
}
Released under the Apache 2.0 license.
5 files, 5.6 GB in total. The weights are 1 file totalling 5.6 GB in pt.
| File | Type | Size | SHA-256 |
|---|---|---|---|
| model_bf16.pt | Weights | 5.6 GB | d5303ac71d28 |
| config.yaml | Configuration | 8.0 KB | — |
| README.md | Documentation | 6.5 KB | — |
| normalizer-relative-10k-pretrain/normalizer.pkl | Other | 10.2 KB | bcae99466048 |
| .gitattributes | Repository | 1.5 KB | — |
Released by EgoSteer through its official repository on Hugging Face. Read the license.
| Precision | Weights in memory |
|---|---|
| As published | 5.6 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Yes. EgoSteer-3B-Base is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
LIBERO 4in1（liberospatial / liberoobject / liberogoal / libero10）共 53.19 GB 的 Wan2.2-VAE 编码 latent 缓存： 训练 LIBERO policy（action head / VLA）时直接读取 latent 缓存，避免重复 VAE 编码。 - 窗口模式: windowed（--windowed） - 输出:.pt 文件，每 episode 一个 - dataset（推荐）: https://huggingface.co/datasets/MangoGoes/libero4in1wan2.2vaelatentdataset - model（本仓库）: https://huggingface.co/MangoGoes/libero4in1wan2.2vaelatentcosmosstyle
Wan2.2-TI2V-5B video DiT + 48-joint action head, trainingmode=joint. The base is suhyeok's finalized B-method recipe: a teacher-forced (sigma=0.25) self-EMA teacher plus an iBOT prototype loss at L18 L18, gamma=0.01, two-view. On top of it the 3 PAST cond latent frames are pooled into one motion frame before a chosen block. These are NOT the surrogate ctxpool runs. The surrogate line (older base, pd8 x GA1) lives in hmkang/wamctxpoolxattn and hmkang/wamctxpoolavg. Do not compare across the two sets. Geometry: 4-latin (numframesin=25, numframesout=41, fdf 2) = 4 cond + 2 future latent slots, 96 tokens per latent frame, 576 tokens per row. Effective batch 16 clips x GA 2 x 2 views = 64 rows…
weighted/imatrix quants of https://huggingface.co/DeepCybo/PhysBrain1.5-8B For a convenient overview and download list, visit our model page for this model. static quants are available at https://huggingface.co/mradermacher/PhysBrain1.5-8B-GGUF This is a vision model - mmproj files (if any) will be in the static repository. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant And here are Artefact2's thoughts on the matter…
static quants of https://huggingface.co/MirilAI/Miril-DroneVLM-2B-2 For a convenient overview and download list, visit our model page for this model. weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion. If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files. (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) Here is a handy graph by ikawrakow comparing some lower-quality quant…
Own lineage: PPO from scratch (nb1, commit 46c48b1) continued as nb2 and nb4 (commit fac88cc), checkpoint 850 (1.328 s); evolution strategies on the weights against the Arena sprint scorer (0.952 s); then the network frozen and wrapped in a step counter carried in the headroll action that adds joint-target and forward-command offsets from about 0.5 m to the line, found by CMA-ES on the event's official seeds: 0.940 s. No other entry's weights used. Simulation only. A perpetual policy for the microduck (61-D observation, 14 actions, 50 Hz). Runs until told otherwise — a gait, loaded into a policy slot. The observation normalizer is baked into policy.onnx; feed raw observations. manifest.json…
EgoSteer-3B-RealMan is the generalist policy from EgoSteer: A Full-Stack System Towards Steerable Dexterous Manipulation from Egocentric Videos. It is a world-model-enhanced Vision-Language-Action (VLA) policy built on a Qwen3-VL backbone with a flow-matching action expert, post-trained from EgoSteer-3B-Base on real-world demonstrations collected on the RealMan robot. This is a robot-ready dual-camera (head + chest) checkpoint, corresponding to EgoSteer-9.6K-DG in the paper. For the base model to fine-tune on your own data, see EgoSteer-3B-Base. Our full-stack system integrates EgoSmith (data pipeline), Robot Stack (deployment), and EgoSteer (policy) to learn from 9.6k hours of large-scale…