Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware

Abstract

Fine manipulation tasks, such as threading cable ties or slotting a battery, are notoriously difficult for robots because they require precision, careful coordination of contact forces, and closed-loop visual feedback. Performing these tasks typically requires high-end robots, accurate sensors, or careful calibration, which can be expensive and difficult to set up. Can learning enable low-cost and imprecise hardware to perform these fine manipulation tasks? We present a low-cost system that performs end-to-end imitation learning directly from real demonstrations, collected with a custom teleoperation interface. Imitation learning, however, presents its own challenges, particularly in high-precision domains: errors in the policy can compound over time, and human demonstrations can be non-stationary. To address these challenges, we develop a simple yet novel algorithm, Action Chunking with Transformers (ACT), which learns a generative model over action sequences. ACT allows the robot to learn 6 difficult tasks in the real world, such as opening a translucent condiment cup and slotting a battery with 80-90% success, with only 10 minutes worth of demonstrations. Project website: https://tonyzhaozh.github.io/aloha/

Full paper on arXiv

Details

arXiv identifier

2304.13705

Published

2023-04-23

Authors

Tony Z. Zhao, Vikash Kumar, Sergey Levine, Chelsea Finn

Models That Cite This Paper

• Described byact_aloha_sim_transfer_cube_human
• Described bychess_single_white_rook_night_tpu_corner_all5_merged
• Described bysoarm_amazing_hand_act
• Described byselect_block_act_octa_4096
• Described byact-finetuned-base-modify-hold-pos
• Described byselect_block_act_fdp3_4096
• Described byselect_block_act_dp3_4096
• Described byfinal_demo_pick_nut_on_scale_20260918_172246_act-policy-v1
• Described by29times