so101-spinning-table-success-head-wrist

This dataset was created using LeRobot.

Dataset Description

SO-101 turntable pick-and-place (robot lil_guy), September 2026.

A single SO-101 follower arm, teleoperated from an SO-101 leader arm, picks a cube off a slowly spinning turntable and places it in a basket. Two cube colours (blue, green) and two operator speeds (slow, fast). Recorded with MakerMods Lab on 2026-09-27 and 2026-09-29.

Features (30 fps): - observation.state: 6 follower joint positions - action: 6 leader joint positions - observation.images.head: Intel RealSense D435if RGB, 640×480, overhead view - observation.images.head_depth: RealSense depth aligned to the head RGB, 12-bit lossless HEVC (decode with lerobot's depth support to get millimetres; values under 20 mm mean no reading) - observation.images.wrist: USB wrist camera, 640×480

The four datasets. This description is shared by four datasets built from the same recordings:

Source sessions:

Selection criteria: - Success: the cube ends in the basket, checked visually on the final frame. There are 3 failures, all with the cube knocked off the turntable. - Clean head camera: - no frozen head frames (at most 2 identical frames in a row) - every depth frame decodes - the two 2026-09-27 episodes with large teleop jumps (over 100 joint units in one frame, from a control-loop stall) are excluded - Clean wrist camera: at most 15% repeated frames, no hold longer than about 0.5 s, and no torn frames. The wrist camera has a failing connector. When it dropped out, the recorder held the camera's last good frame, so wrist streams outside the clean set contain frozen stretches.

Known issues: - 2026-09-27 session: - The head camera sat slightly differently; it was repositioned before 2026-09-29. - The wrist camera ran at about 16–20 fps, so 18–39% of its frames are repeats. - Episodes 7 and 14 contain the teleop jumps. - Episode 3's first depth frame fails random-access decoding. It is only in the all dataset. - The first head RGB frame of each 2026-09-29 session is overexposed while the RealSense exposure settles. - Task strings differ slightly between sessions, as typed at record time.

Episode provenance (new episode index ← source session episode indices):

• so101-spinning-table-all
• episodes 0–14 ← so-101-poc-blue-slow_20260927_194226 episodes [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]
• episodes 15–24 ← blue_slow_20260929_145703 episodes [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
• episodes 25–39 ← green_slow_20260929_150720 episodes [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]
• episodes 40–54 ← green_fast_20260929_151319 episodes [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]
• episodes 55–68 ← blue_fast_20260929_152142 episodes [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]
• so101-spinning-table-success-head
• episodes 0–11 ← so-101-poc-blue-slow_20260927_194226 episodes [0, 1, 2, 4, 5, 6, 8, 9, 10, 11, 12, 13]
• episodes 12–20 ← blue_slow_20260929_145703 episodes [0, 1, 2, 4, 5, 6, 7, 8, 9]
• episodes 21–35 ← green_slow_20260929_150720 episodes [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]
• episodes 36–48 ← green_fast_20260929_151319 episodes [1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]
• episodes 49–62 ← blue_fast_20260929_152142 episodes [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]
• so101-spinning-table-success-head-wrist
• episodes 0–2 ← blue_slow_20260929_145703 episodes [0, 2, 5]
• episodes 3–13 ← green_slow_20260929_150720 episodes [0, 1, 3, 4, 5, 6, 9, 10, 11, 13, 14]
• episodes 14–17 ← green_fast_20260929_151319 episodes [1, 4, 7, 8]
• episodes 18–24 ← blue_fast_20260929_152142 episodes [0, 1, 2, 3, 6, 7, 10]
• so101-spinning-table-failures
• episodes 0–0 ← blue_slow_20260929_145703 episodes [3]

• episodes 1–2 ← green_fast_20260929_151319 episodes [0, 3]

• Homepage: [More Information Needed]

• Paper: [More Information Needed]
• License: apache-2.0

Dataset Structure

meta/info.json:

{
    "codebase_version": "v3.0",
    "fps": 30,
    "features": {
        "action": {
            "dtype": "float32",
            "shape": [
                6
            ],
            "names": [
                "shoulder_pan.pos",
                "shoulder_lift.pos",
                "elbow_flex.pos",
                "wrist_flex.pos",
                "wrist_roll.pos",
                "gripper.pos"
            ]
        },
        "observation.state": {
            "dtype": "float32",
            "shape": [
                6
            ],
            "names": [
                "shoulder_pan.pos",
                "shoulder_lift.pos",
                "elbow_flex.pos",
                "wrist_flex.pos",
                "wrist_roll.pos",
                "gripper.pos"
            ]
        },
        "observation.images.wrist": {
            "dtype": "video",
            "shape": [
                480,
                640,
                3
            ],
            "names": [
                "height",
                "width",
                "channels"
            ],
            "info": {
                "is_depth_map": false,
                "video.height": 480,
                "video.width": 640,
                "video.codec": "h264",
                "video.pix_fmt": "yuv420p",
                "video.fps": 30,
                "video.channels": 3,
                "has_audio": false,
                "video.g": 2,
                "video.crf": 30,
                "video.preset": "ultrafast",
                "video.fast_decode": 0,
                "video.video_backend": "pyav",
                "video.extra_options": {}
            }
        },
        "observation.images.head": {
            "dtype": "video",
            "shape": [
                480,
                640,
                3
            ],
            "names": [
                "height",
                "width",
                "channels"
            ],
            "info": {
                "is_depth_map": false,
                "video.height": 480,
                "video.width": 640,
                "video.codec": "h264",
                "video.pix_fmt": "yuv420p",
                "video.fps": 30,
                "video.channels": 3,
                "has_audio": false,
                "video.g": 2,
                "video.crf": 30,
                "video.preset": "ultrafast",
                "video.fast_decode": 0,
                "video.video_backend": "pyav",
                "video.extra_options": {}
            }
        },
        "observation.images.head_depth": {
            "dtype": "video",
            "shape": [
                480,
                640,
                1
            ],
            "names": [
                "height",
                "width",
                "channels"
            ],
            "info": {
                "is_depth_map": true,
                "depth_unit": "mm",
                "video.height": 480,
                "video.width": 640,
                "video.codec": "hevc",
                "video.pix_fmt": "gray12le",
                "video.fps": 30,
                "video.channels": 1,
                "has_audio": false,
                "video.g": 2,
                "video.crf": 30,
                "video.preset": null,
                "video.fast_decode": 0,
                "video.video_backend": "pyav",
                "video.extra_options": {
                    "x265-params": "lossless=1"
                },
                "video.depth_min": 0.01,
                "video.depth_max": 10.0,
                "video.shift": 3.5,
                "video.use_log": true
            }
        },
        "timestamp": {
            "dtype": "float32",
            "shape": [
                1
            ],
            "names": null
        },
        "frame_index": {
            "dtype": "int64",
            "shape": [
                1
            ],
            "names": null
        },
        "episode_index": {
            "dtype": "int64",
            "shape": [
                1
            ],
            "names": null
        },
        "index": {
            "dtype": "int64",
            "shape": [
                1
            ],
            "names": null
        },
        "task_index": {
            "dtype": "int64",
            "shape": [
                1
            ],
            "names": null
        }
    },
    "total_episodes": 25,
    "total_frames": 8830,
    "total_tasks": 3,
    "chunks_size": 1000,
    "data_files_size_in_mb": 100,
    "video_files_size_in_mb": 200,
    "data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
    "video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4",
    "robot_type": "so_follower",
    "splits": {
        "train": "0:25"
    }
}

Citation

BibTeX:

[More Information Needed]