Source-linked AI summary
LeRobot: An Open-Source Library for End-to-End Robot Learning
Remi Cadene, Simon Aliberts, Francesco Capuano, Michel Aractingi, Adil Zouitine, Pepijn Kooijmans, Jade Choghari, Martino Russi, Caroline Pascal, Steven Palma, Mustafa Shukor, Jess Moss, Alexander Soare, Dana Aubakirova, Quentin Lhoest, Quentin Gallouédec, Thomas Wolf
TL;DR
Robot learning research is slowed by fragmented tools, interfaces, and data formats that hinder reproducibility and practical reuse. LeRobot provides an open, vertically integrated stack for robot control, scalable datasets, inference, and learning algorithms; its community ecosystem has produced thousands of datasets, hundreds of thousands of episodes, and hundreds of openly contributed models. Its current robot and algorithm coverage remains incomplete.
Problem
Fragmented middleware, dataset formats, and learning frameworks hinder reproducibility and raise the barrier to entry for robot learning research.
Method
LeRobot unifies real-world robot middleware, standardized multimodal datasets, decoupled inference, and PyTorch robot-learning algorithms in an open end-to-end stack.
Results
Thousands of datasets, hundreds of thousands of episodic data, and hundreds of openly contributed models have been produced through the LeRobot community ecosystem.
Takeaways & Limitations
LeRobot provides a shared platform for collecting, reusing, training on, and deploying real-world robot-learning data and models across supported platforms.
Takeaways & Limitations
Robot and robot-learning algorithm coverage is currently non-exhaustive, with extending support remaining future work.
Abstract
from arXiv · showhide
Robotics is undergoing a significant transformation powered by advances in high-level control techniques based on machine learning, giving rise to the field of robot learning. Recent progress in robot learning has been accelerated by the increasing availability of affordable teleoperation systems, large-scale openly available datasets, and scalable learning-based methods. However, development in the field of robot learning is often slowed by fragmented, closed-source tools designed to only address specific sub-components within the robotics stack. In this paper, we present \texttt{lerobot}, an open-source library that integrates across the entire robot learning stack, from low-level middleware communication for motor controls to large-scale dataset collection, storage and streaming. The library is designed with a strong focus on real-world robotics, supporting accessible hardware platforms while remaining extensible to new embodiments. It also supports efficient implementations for various state-of-the-art robot learning algorithms from multiple prominent paradigms, as well as a generalized asynchronous inference stack. Unlike traditional pipelines which heavily rely on hand-crafted techniques, \texttt{lerobot} emphasizes scalable learning approaches that improve directly with more data and compute. Designed for accessibility, scalability, and openness, \texttt{lerobot} lowers the barrier to entry for researchers and practitioners to robotics while providing a platform for reproducible, state-of-the-art robot learning.
1 INTRODUCTION
Robot learning is shifting from difficult-to-scale explicit, hand-crafted pipelines toward data-driven approaches, but fragmented tools still hinder reproducibility and access. LeRobot addresses this gap with an open, vertically integrated stack spanning robot integration, datasets, inference, and algorithms.
- Motivation: Explicit robotics models require substantial expert effort and are difficult to derive accurately for diverse deployment scenarios, limiting scalability.The paper contrasts these models with learned approaches that can improve with more data and compute.
- Motivation: The robot learning ecosystem remains fragmented because robot-specific control interfaces and incompatible dataset tooling make contributions difficult to reproduce and use.These issues affect both middleware adaptation and data reuse across platforms and tasks.
- Unified stack: LeRobot provides a consistent Python middleware API for real-world motor control across platforms ranging from low-end manipulators to humanoid arms and hands.The middleware bridges machine-learning frameworks with real-world robotics across diverse robot embodiments.
- Unified stack: LeRobot standardizes multimodal, high-frame-rate data recording, storage, and streaming through the scalable LeRobotDataset format.Its integration with the open-source ecosystem is intended to support openness and research reproducibility.
- Unified stack: The library decouples action planning from control execution physically and logically, enabling remote and asynchronous inference alongside low-level control loops.This design supports additional computation and dynamic runtime adaptation.
- Unified stack: LeRobot includes PyTorch implementations of state-of-the-art robot-learning methods for training custom models and using openly available pre-trained models.The library supports multiple learning paradigms while allowing models to be trained or reused within the same stack.
- Contribution: Vertical integration across the stack reduces the barrier to entry while emphasizing accessibility and scalability.The paper presents this integration as an effort to address fragmentation and accelerate progress.
2 BACKGROUND
Robot learning replaces many hand-engineered, modular robotics components with learned models that can adapt through interaction data, while affordable teleoperation and open hardware support broader data collection. The field still faces fragmentation in middleware, datasets, learning frameworks, and reproducibility, motivating unified tooling.
- 2.1 EXPLICIT AND IMPLICIT MODELS: Classical robotics uses modular explicit models for perception, planning, and control, whereas robot learning uses monolithic implicit policies learned from interaction data.The paper frames this as a shift from hand-crafted components toward learned representations.
- 2.1 EXPLICIT AND IMPLICIT MODELS: Explicit-model pipelines can suffer compounding errors, poor scalability, and undermodeling of physical interactions in unstructured environments.Their effectiveness is described as more limited in settings such as houses than factory lines.
- 2.1 EXPLICIT AND IMPLICIT MODELS: Learned implicit models are described as more adaptable because their performance empirically scales with larger datasets and more compute.Robot learning policies can learn end-to-end from interaction data rather than relying on hand-tuned pipeline components.
- 2.1 EXPLICIT AND IMPLICIT MODELS: Affordable teleoperation hardware enables expert demonstrations to be collected for real-world robot learning.Teleoperation assigns cognitive decisions to the human while the robot performs their mechanical implementation.
- 2.3 PRACTICAL CHALLENGES FOR ROBOT LEARNING RESEARCH: Robot learning algorithms often remain standalone components whose integration with the broader robotics stack is challenging.This integration problem contributes to ecosystem fragmentation and makes research harder to reproduce.
- 2.3 PRACTICAL CHALLENGES FOR ROBOT LEARNING RESEARCH: Platform-specific middleware, heterogeneous dataset formats, and hardware-dependent implementation differences hinder reproducibility and data aggregation.Examples of dataset formats include TensorFlow Datasets, ROS bags, and bespoke JSON layouts.
- 2.3 PRACTICAL CHALLENGES FOR ROBOT LEARNING RESEARCH: LeRobot addresses ecosystem-wide incidental complexity by unifying hardware interfacing, data collection and streaming, and policy training and deployment.The stated goal is to reduce engineering overhead while providing an open and scalable library.
3 FEATURES
lerobot integrates accessible robot hardware, standardized and streamable datasets, asynchronous inference, and reusable learning algorithms into one end-to-end stack. Its features target scalable, reproducible robot learning across platforms, data modalities, computational settings, and model families.
- Accessible real-world robots: lerobot supports multiple real-world robot platforms through shared, extensible middleware that interfaces with low-cost actuator SDKs.The supported platforms include static and mobile manipulators, humanoid arms, hands, and mobile manipulation systems.
- Datasets: 16K+ datasets from 2.2K+ contributors use the unified LeRobotDataset multimodal schema as of September 2025.The schema covers sensorimotor readings, camera feeds, teleoperation signals, task descriptions, embodiment details, and capture parameters.
- Datasets: LeRobotDataset is designed for large-scale sequential multimodal data and integrates with PyTorch to support standardized, repeatable research workflows.Its architecture targets repositories potentially containing millions of expert trajectories.
- Datasets: Native streaming lets users process remotely hosted datasets without downloading the entire corpus, improving accessibility for large-scale robot learning.Streaming fetches frames on demand rather than preloading the full dataset into local memory.
- Models: lerobot provides PyTorch implementations spanning reinforcement learning, single-task and multi-task behavioral cloning, custom models, and openly available pretrained models.Supported examples include HIL-SERL, TD-MPC, ACT, Diffusion Policy, VQ-BET, π0, and SmolVLA.
- Inference: The inference stack decouples action prediction from control physically and logically, enabling remote computation and asynchronous execution of action chunks.All supported behavioral-cloning policies predict action chunks; control continues at the desired frequency while inference runs separately.
4 SIMULATION
lerobot supports simulation-based benchmarking through LIBERO and Meta-World, while emphasizing real-world training because simulation is challenging for contact-rich tasks.
- 4 SIMULATION: Simulation is used primarily to evaluate robot learning algorithms, while training relies as much as possible on real-world data.The paper identifies simulation as challenging for the contact-rich, complex tasks targeted by lerobot.
- 4 SIMULATION: lerobot provides API-level evaluation support for both LIBERO and Meta-World, widely used simulation benchmarks in robot learning research.
- LIBERO: LIBERO contains four fixed task suites with 10 tasks each, varying spatial arrangement, object, and overall task conditions.Typical evaluation reports success rate over test episodes, and lerobot natively integrates LIBERO.
- Meta-World: Meta-World contains 50 robotic manipulation tasks organized into multi-task and meta-learning suites for evaluating simultaneous learning and adaptation to new tasks.All tasks use the same robotic arm and setup while varying objects and task combinations.
5 CONCLUSIONS
lerobot is presented as a unified, open-source end-to-end stack connecting robot control, large-scale data tooling, and scalable learning algorithms. Its standardized data infrastructure supports reuse at scale, while incomplete hardware and algorithm coverage and missing low-level inference optimizations remain limitations.
- 5 CONCLUSIONS: lerobot bridges low-level control, large-scale data tooling, and scalable learning algorithms in a unified open-source stack.
- 5 CONCLUSIONS: Shared middleware enables accessible teleoperation and real-world data collection across multiple robot platforms.
- 5 CONCLUSIONS: Standardized datasets support data collection and reuse at scale, yielding thousands of datasets, hundreds of thousands of episodes, and hundreds of openly contributed models.
- Limitations: Current support covers a practical but incomplete set of robots, sensors, controllers, and robot learning algorithms.Strong practical inference performance also still requires low-level optimizations such as quantization and graph compilation, which the library currently disregards.
B REAL-WORLD ROBOTS API
The real-world robots API section includes implementation examples and references related dataset and robot-platform resources.
- B REAL-WORLD ROBOTS API: The API examples import teleoperator interfaces for SO-100 leader and follower robots.
- B REAL-WORLD ROBOTS API: The example notes that compatible teleoperation setups can also include Reachy-2 and LeKiwi.
- B REAL-WORLD ROBOTS API: The section references openly available datasets using the LeRobotDataset format and a table breaking down the Other robot category.
- B REAL-WORLD ROBOTS API: The example sends an action through robot.send_action(action) to move the robot according to the action.
C DATASETS
The Other dataset category is dominated by faulty datasets that omit the recorded robot platform, while the most downloaded datasets demonstrate adoption of the LeRobotDataset format beyond natively integrated platforms.
- C DATASETS: Datasets tagged as unknown dominate the Other category because they do not explicitly record the robot platform used.
- C DATASETS: The largest number of downloads belongs to a platform not natively integrated in lerobot, indicating adoption of the LeRobotDataset format across the robotics community.
C.1 STREAMING DATASETS
StreamingLeRobotDataset addresses the difficulty of using million-episode robotic datasets by fetching high-volume data on demand rather than pre-loading it locally. Its design uses sequential streaming and supports multi-frame retrieval for robot-learning pipelines.
- StreamingLeRobotDataset targets dataset handling at the million-episodes scale, where pre-loading data into local memory becomes impractical.
- The streaming paradigm fetches individual frames on demand instead of requiring the full dataset to be downloaded before use.
- The streaming design supports robust learning and multi-frame retrieval despite an inherently sequential, non-indexable setting.
- LeRobotDataset stores robotic data in tabular Parquet records, compressed MP4 videos, and lightweight metadata.
- Metadata is downloaded fully, while high-volume video and control streams are processed on demand.
C.2 EXAMPLE: USE A DATASET
The dataset example shows how LeRobotDataset supplies indexed observations and actions, including wrist-camera images and temporal context, for DataLoader-based training or inference. StreamingLeRobotDataset can alternatively avoid downloading the dataset.
- Temporal context can be specified with values such as -0.2, -0.1, and 0.0 seconds before each frame.
- StreamingLeRobotDataset is presented as an option for avoiding dataset downloads.
- The example uses the lerobot/svla_so101_pickplace dataset.
- LeRobotDataset exposes indexed dataset frames containing observation state, actions, and wrist-camera image tensors.
- The dataset can be wrapped in a DataLoader for training or inference and iterated batch by batch across epochs.
C.3 EXAMPLE: USE A STREAMING DATASET
The streaming-dataset example imports StreamingLeRobotDataset and uses it to stream frames on the fly without downloading the dataset. The example applies this to lerobot/svla_so101_pickplace.
- The example imports StreamingLeRobotDataset from LeRobot’s streaming-dataset module.
- StreamingLeRobotDataset streams frames on the fly without downloading the dataset.
- The demonstrated dataset is lerobot/svla_so101_pickplace.
D.1 EXAMPLE: TRAIN A MODEL
The training example combines LeRobot dataset utilities, feature handling, preprocessing, and pluggable policies with training-loop and robot-control components. It also presents asynchronous inference as a way to reduce cycle times while maintaining comparable performance, with network access required for the scalable remote-inference path.
- The workflow derives policy features from dataset features and constructs preprocessing and postprocessing components.
- LeRobot provides plug-in policies, including DiffusionPolicy and SmolVLAPolicy, within a common training workflow.
- Training iterates over dataloader batches, reports loss, stops at the configured training step, and saves policy-related checkpoints.
- The example preprocesses observations, selects and postprocesses actions, then sends the resulting action to the robot.
- Asynchronous inference reduces cycle times and increases throughput while maintaining similar success rates across three SO-100 tasks over 10 test episodes.
- The asynchronous speedup comes from computing the next action chunk while the current chunk is still being executed, and the scalable path requires network access.
- The remote-inference example separates a policy server from a robot client, with the client attaching to the server and running a control loop.