Gen-HumanEgo

1,800+ hours of egocentric human demonstrations with synchronized, structured supervision for embodied AI and robot learning.

Gen-HumanEgo contains real-world first-person demonstrations collected across diverse tasks, people, environments, and ways of performing activities using a unified six-camera DAS-Ego setup. GenRobot's Data Foundation Model (DFM) processes the recordings to provide complementary training signals for human motion, spatial understanding, and task understanding.

The dataset includes:

• Egocentric RGB: synchronized multi-view video recordings.
• 3D hand reconstruction: 3D hand keypoints, MANO parameters, and full hand geometry.
• Spatial understanding: large-field-of-view Ego-Depth.
• Semantic understanding: hierarchical video-, task-, and subtask-level annotations with temporal boundaries.

For a detailed introduction to the dataset, see our Gen-HumanEgo Technical Blog

Dataset at a glance

Repository structure

The task-organized directory tree follows the dataset's scenario/skill hierarchy. The final two levels are a two-character file-prefix directory and the episode's MCAP filename.

Gen-HumanEgo/
├── README.md
├── egov4_urdf.zip
├── assets/
│   └── ...
└── data/
    ├── industry/
    │   └── logistics/
    │       └── sorting_and_packing/
    │           └── daily_work/
    │               └── 00/
    │                   └── 005c469cf7424c4c87054569b2234ebc.mcap
    ├── domestic_services/
    │   └── living_room/
    │       └── clothing_organization/
    │           └── iron_clothes/
    │               └── c0/
    │                   └── c096609eca54434e9fa93b0fff55ce3d.mcap
    └── business/
        └── restaurant/
            └── bakery/
                └── daily_work/
                    ├── a9/
                    │   └── a91e57586c33437292d763fb8fd3777c.mcap
                    └── 10/
                        └── 101207375b8e41fea31bacd94db480e6.mcap

What's inside an episode?

Each episode is packaged as an MCAP recording. The release includes synchronized video and structured outputs, summarized below.

The hierarchical annotation design describes activity at three levels:

1. Video: an overall description of the episode.
2. Task: the objective of a continuous segment, with time/frame boundaries and scene information.
3. Subtask: a more detailed action caption, time/frame boundaries, and a success field (is_success), with objects, observable attributes, and spatial relationships expressed in the caption when applicable.

For example, an episode about tidying a sofa can include the task “Tidy up the sofa” and subtasks such as “Place the red and black throw pillow on the sofa back” and “Flatten the sofa cover on the right seat of the sofa.”

Documentation and tools

• GenRobot DAS-Ego Data Introduction — topic definitions, message schemas, coordinate systems, timestamp conventions, and transformation definitions. GenRobot DAS-Ego Data Introduction
• GenRobot MCAP Visualization Tool — browser-based visualization of MCAP episodes. GenRobot MCAP Visualization Tool
• GenRobot DAS DataKit — tools for reading, validating, visualizing, and converting DAS-Ego data. GenRobot DAS DataKit
• Ego URDF — egov4_urdf.zip, containing the DAS-Ego device link structure and sensor-frame definitions.

Contact

Questions, suggestions, and requests for additional scenarios or skills are welcome.

• X: @GenrobotAI
• LinkedIn: GenRobot
• Email: [email protected]
• Discord: https://discord.gg/rSSb5thgu