Robot Learning with LeRobot 2026
Practical guide to imitation learning and reinforcement learning for robot manipulation — LeRobot ACT, Diffusion Policy, Isaac Lab, Open-X Embodiment, and π0.
TOP RECOMMENDATION 2026
Start with LeRobot + ACT on SO-ARM100
$100 arm + 50 demos + 2h training = working manipulation policy. Fastest path from zero to real robot.
Method Comparison
| Method | Demos needed | Train time | Sim-to-real | Language | Rating |
|---|---|---|---|---|---|
| ACT (LeRobot) | 50–200 | 1–4h | ❌ | ❌ | ★★★★★ |
| Diffusion Policy | 50–300 | 4–12h | ❌ | ❌ | ★★★★☆ |
| Isaac Lab (PPO) | 0 (RL) | 30m–4h | ✅ | ❌ | ★★★★☆ |
| π0 fine-tune | 50–200 | 2–8h | ❌ | ✅ | ★★★★☆ |
| OXE + Octo | 50–500 | 4–24h | ❌ | ✅ | ★★★★☆ |
Framework Reviews
LeRobot (ACT)
HuggingFace · imitation
95/100
HuggingFace's robotics framework built around Action Chunking with Transformers (ACT). Collect demos → train → deploy on real hardware in hours.
Hardware
Low-cost SO-ARM100 ($100) or Franka FR3
Train time
1–4 hours (GPU) for 50 demos
Data needed
50–200 teleoperation demos
✓ Pros
- ✓Fastest path from idea to working robot policy
- ✓End-to-end pipeline: teleoperation → dataset → train → eval
- ✓SO-ARM100 ($100 kit) makes real robot learning accessible to everyone
- ✓HuggingFace Hub integration — share datasets and pre-trained policies
- ✓Active community: 100+ robot policy datasets on lerobot-community
- ✓ACT checkpoint zoo: pretrained policies for common manipulation tasks
- ✓Python-first API: lerobot.common.policies.act
✗ Cons
- ✗ACT needs 50+ quality demos — data collection is the bottleneck
- ✗Distribution shift: policies fail on environments they weren't trained in
- ✗No sim-to-real transfer built-in (unlike Isaac Lab)
Diffusion Policy
Columbia / MIT · imitation
91/100
Diffusion-based behavior cloning that models action distributions with a denoising diffusion process. More expressive than ACT for multimodal tasks.
Hardware
Any ROS 2 compatible arm with camera
Train time
4–12 hours (50 demos, RTX 4090)
Data needed
50–300 demos
✓ Pros
- ✓Handles multimodal action distributions — multiple valid strategies per task
- ✓More stable convergence than vanilla behavior cloning
- ✓UNet and Transformer diffusion variants available
- ✓Compatible with LeRobot dataset format
- ✓Strong results on Push-T, Block Stacking, Can Picking benchmarks
✗ Cons
- ✗Inference slower (10–20ms per step) due to denoising iterations
- ✗More hyperparameters than ACT — harder to tune
- ✗Memory intensive: 16GB+ VRAM for transformer variant
Isaac Lab
NVIDIA · rl
92/100
NVIDIA's GPU-parallelized robot learning framework. Train PPO/SAC policies on 4096 parallel environments in minutes. Best-in-class sim-to-real transfer.
Hardware
NVIDIA GPU (8GB+ VRAM) + Omniverse
Train time
30 min – 4 hours (4096 envs)
Data needed
No demos needed (RL from scratch)
✓ Pros
- ✓4096 parallel environments: 100× faster than single-env RL
- ✓Best sim-to-real transfer in the field — tight Omniverse physics
- ✓Stable Baselines 3 / SKRL / RSL-RL integration
- ✓Domain randomization built-in: friction, mass, camera pose, lighting
- ✓IsaacGym-compatible migration path (IsaacGym → Isaac Lab official)
- ✓Franka, UR, Unitree, ANYmal built-in robot models
✗ Cons
- ✗Requires NVIDIA GPU + Omniverse license setup (complex)
- ✗Sim-to-real gap remains for contact-rich tasks
- ✗RL still harder to get working than imitation learning for manipulation
Open-X Embodiment
Google DeepMind + 33 labs · dataset
89/100
22 million robot trajectories across 527 skills from 22 robot types. The ImageNet of robot learning — fine-tune RT-2, Octo, or π0 on your task.
Hardware
Any arm (for fine-tuning)
Train time
4–24 hours fine-tuning
Data needed
50–500 demos for fine-tuning on top
✓ Pros
- ✓22M trajectories — largest robot dataset in existence
- ✓Enables few-shot fine-tuning: 50 demos on custom task
- ✓Models trained on OXE transfer to new robots better than from scratch
- ✓Octo and π0 pretrained on OXE — download and fine-tune
- ✓Standardized RLDS format compatible with LeRobot, TF, PyTorch
✗ Cons
- ✗OXE dataset is 1.2TB+ — storage and bandwidth are real constraints
- ✗Quality varies significantly across contributing labs
- ✗Fine-tuned OXE models still lag behind task-specific from-scratch for simple tasks
π0 (Pi-Zero)
Physical Intelligence · foundation
90/100
Physical Intelligence's π0 — a flow matching-based generalist robot policy pre-trained on diverse data. Fine-tunable to new tasks with ~100 demos.
Hardware
Franka, UR, custom arms
Train time
2–8 hours fine-tuning
Data needed
50–200 demos
✓ Pros
- ✓Best generalization across task types of any current foundation model
- ✓Flow matching: faster inference than diffusion (single forward pass)
- ✓Handles dexterous manipulation, bimanual, mobile manipulation
- ✓Open weights announced for π0.5 — community access growing
- ✓Language conditioning: 'fold the towel' → action sequence
✗ Cons
- ✗Open weights only for π0.5 — π0 still closed
- ✗Compute intensive: 7B+ parameter model
- ✗Fine-tuning infrastructure less mature than LeRobot
LeRobot Quickstart
End-to-end: install → download dataset → train ACT → evaluate → collect your own demos.
Install LeRobot
# Clone and install git clone https://github.com/huggingface/lerobot cd lerobot pip install -e ".[feetech]" # for SO-ARM100 (Feetech servos) # pip install -e ".[dynamixel]" # for Trossen/Dynamixel arms # pip install -e ".[franka]" # for Franka FR3
Download a community dataset
from lerobot.common.datasets.lerobot_dataset import LeRobotDataset
# Download the lerobot-community/so100_pick_place_lego dataset
dataset = LeRobotDataset("lerobot-community/so100_pick_place_lego")
print(f"Episodes: {dataset.num_episodes}")
print(f"Frames: {dataset.num_frames}")
print(f"Tasks: {dataset.tasks}")
print(f"Features: {dataset.features}")Train an ACT policy
# Train ACT on the downloaded dataset python lerobot/scripts/train.py \ policy=act \ env=aloha \ dataset_repo_id=lerobot-community/so100_pick_place_lego \ hydra.run.dir=outputs/train/act_lego \ training.num_epochs=100 \ training.batch_size=64 \ training.save_checkpoint=true # Training output: outputs/train/act_lego/checkpoints/
💡 Set training.batch_size=32 if you have less than 12GB VRAM.
Evaluate in simulation
# Evaluate in gym_pusht simulation python lerobot/scripts/eval.py \ -p outputs/train/act_lego/checkpoints/last/pretrained_model \ eval.n_episodes=10 \ eval.batch_size=10 # Output: success_rate, mean_reward, episode videos
Collect your own demonstrations
# Record demos on real SO-ARM100 hardware python lerobot/scripts/control_robot.py record \ --robot-path lerobot/configs/robot/so100.yaml \ --fps 30 \ --repo-id your-username/my_task_demos \ --tags tutorial \ --warmup-time-s 5 \ --episode-time-s 40 \ --reset-time-s 10 \ --num-episodes 50 # Demos saved to HuggingFace Hub automatically
💡 50 demos of 30–40s each ≈ 2.5 hours of collection time. Quality > quantity — discard failed demos.