Robot Learning: A Tutorial

1 Introduction

• LeRobot integrates tools across the robotics stack, including broad platform support, a standardized dataset format, RL and BC algorithms, and an inference stack that separates action planning from execution.The tutorial aims to give researchers and practitioners an intuitive, concise account of robot-learning concepts alongside practical code examples.Source: §1 · pp. 3–4

1.1 LeRobotDataset

• LeRobotDataset standardizes multimodal robotics data, combining sensorimotor readings, camera feeds, teleoperation status, task descriptions, robot type, and measurement details.It provides a customizable interface integrated with PyTorch and Hugging Face and supports data from varied robot embodiments and simulations.Source: §1.1 · p. 4

1.1.1 The dataset class design

• LeRobotDataset separates storage from its user-facing API and organizes data as tabular streams, grouped camera videos, and relational metadata for efficient access and scalable episode reconstruction.The design concatenates data from multiple episodes to limit filesystem pressure, relying on metadata to recover episode boundaries.Source: §1.1.1 · p. 4

1.2 Code Example: Batching a (Streaming) Dataset

• LeRobotDataset provides time-window sampling around frames, padding unavailable frames and returning a mask, while supporting batched loading and streaming large Hub datasets for training.Streaming supports high-performance batch processing at approximately 80–100 it/s, depending on connectivity, without requiring users to download the full dataset.Source: §1.2 · p. 5

1.3 Code Example: Collecting Data

• Motion-generation methods are grouped by whether they explicitly model robot–environment interactions (dynamics-based) or implicitly model them (learning-based).The overview is not exhaustive, as noted with reference to Bekris et al. (2024).Source: Figure 2 · p. 9

2 Classical Robotics

• Robotics methods range from explicit, mechanics-based models to implicit, learning-based models, with approaches also combining learned methods and physical models.The tutorial frames its aim as explaining why these ideas have converged into modern robot learning, rather than providing comprehensive robotics or machine-learning coverage.Source: §2.1 · p. 9

2.2 Different Types of Motion

• Robotics motion can alter the environment, the robot’s relative location, or both, and learning has complemented traditional methods across the robotics stack.The section describes learning both as improving state estimation for planning and as replacing controllers with perception-to-action methods.Source: §2.2 · p. 10
• Manipulation changes the environment, locomotion changes the robot’s location, and mobile manipulation combines both with a larger control-variable set.Learning-based approaches have also been studied across locomotion, whole-body control, and manipulation, alongside explicit models.Source: §2.2 · p. 10

2.3 Example: Planar Manipulation

• In the planar SO-100 example, inverse kinematics finds a configuration for a target pose, but not the trajectory; differential inverse kinematics instead solves for joint velocities to track target motion.Obstacles constrain feasible configurations, and differential inverse kinematics depends on specified target velocities and access to the Jacobian and its pseudoinverse, assumptions that can be challenging in dynamic settings or complex chains.Source: §2.3 · pp. 12–13

2.3.1 Adding Feedback Loops

• Feedback control can mitigate modeling errors, sensing noise, and disturbances, but gain tuning is system-specific and intermittent contact can challenge stability guarantees.The example modifies joint-velocity commands using the error between target and measured pose; manipulation contacts can induce hybrid dynamics and Jacobian discontinuities.Source: §2.3.1 · p. 13

2.4 Limitations of Dynamics-based Robotics

• Dynamics-based pipelines often require task- and platform-specific expertise, while their modular interfaces and compact state representations complicate adaptation and multimodal, multitask scaling.Reworking objectives, constraints, and heuristics across pipeline stages adds engineering overhead; approximations of contact, friction, and compliance can also limit real-world applicability.Source: §2.4 · p. 14
• The section motivates learning-based robotics as a way to integrate perception and control, adapt across tasks and embodiments with less expert modeling, and use growing robotics data.Classical approaches may not maximally benefit from increasingly available datasets contributed by centralized groups and decentralized individuals.Source: §2.4 · p. 15

3 Robot (Reinforcement) Learning

• Generalist language-conditioned models are grouped with task-specific behavioral-cloning methods because both largely learn to reproduce trajectories from demonstrations, despite differing in task breadth and language conditioning.The tutorial presents both categories among the robot-learning policies implemented in lerobot.Source: §3 · p. 17

3.1 A (Concise) Introduction to RL

• RL agents improve through trial and error using rewards, and MDPs represent stochastic, discrete-time interactions while supporting learning without explicit dynamics models.The tutorial focuses on finite-horizon episodic MDPs; states can include robot configuration, velocities, and sensor readings, while actions can be torques, velocities, or end-effector commands.Source: §3.1 · pp. 17–18

3.2 Real-world RL for Robotics

• Real-world RL faces risky exploratory actions, costly manual resets, and high data demands; simulation improves safety and throughput but creates reality-gap and fidelity limitations.Contact-rich tasks such as folding deformable materials can be too time-intensive to simulate at useful fidelity.Source: §3.2 · pp. 20–21
• Domain randomization varies simulated dynamics to improve transfer, but choosing relevant parameters and randomization strength is task-specific; too little risks transfer failure, while too much can hurt performance.The support of the randomization distribution may omit important factors, and excessive randomization can over-regularize learning.Source: §3.2 · pp. 20–21
• Replay-buffer off-policy RL can reuse transitions and incorporate prior data; demonstrations and human interventions support real-world learning, with near-perfect manipulation success reported in 1–2 hours.The cited human-intervention result concerns challenging manipulation tasks.Source: §3.2 · p. 21
• DQN uses neural Q-functions for large, unstructured state spaces, while DDPG extends deterministic policy gradients to continuous actions and high-dimensional observations.DDPG also reuses past transitions through a replay buffer.Source: §3.2 · pp. 22–23
• SAC’s MaxEnt objective encourages diverse behavior through policy-entropy regularization, while RLPD trains from scratch using equal offline and online replay samples.RLPD also uses LayerNorm to prevent value overestimation and ensembles to form TD-targets.Source: §3.2 · p. 23
• SERL learns reward classifiers from success and failure demonstrations for hard-to-specify dynamic tasks, and pairs forward task-completion controllers with backward reset controllers.The approach also frames proprioceptive configurations and actions relative to the initial end-effector pose, then randomizes that initial pose to improve goal robustness and scalability.Source: §3.2 · pp. 23–24
• HIL-SERL achieves near-perfect success rates (99%+) on diverse manipulation tasks within 1-2 hours, using offline demonstrations and targeted online human corrections.Human intervention data is stored in both offline and online replay buffers, providing stronger supervision while allowing autonomous learning.Source: §3.2 · p. 24

3.2.1 Code Example: Real-world RL

• The LeRobot HIL-SERL example separates environment interaction and transition collection in an Actor from a Learner that updates policy parameters using equal online and offline batches.The implementation can run locally or across network-connected machines.Source: §3.2.1 · pp. 24–25, Figure 17 · p. 25
• The learner trains the SAC policy on transitions streamed from the actor and sends updated policy parameters back for the actor to adopt.The code describes parameter updates every 10 training steps.Source: §3.2.1 · pp. 28–29
• The learner adds all incoming transitions to an online buffer, stores intervention transitions in an offline buffer, and combines samples from both for SAC training.The code labels this combined-batch approach the key HIL-SERL mechanism.Source: §3.2.1 · p. 29
• The example configures an SO100 follower and leader, loads a pretrained reward classifier, and initializes demonstrations from a LeRobot dataset.Source: §3.2.1 · pp. 30–31
• The real-world RL code example preloads an offline replay buffer from demonstrations and runs separate learner and actor processes with device-specific settings.The learner can use accelerated hardware, while the frozen actor can run on CPU or be accelerated; the example also includes graceful shutdown handling.Source: §3.2.1 · p. 31

3.2.2 Limitations of RL in Real-World Robotics: Simulators and Reward Design

• High-fidelity simulators are difficult to build and maintain for contact-rich manipulation and soft-material tasks, even when simulation is the only viable option.Simulation may be necessary when gathering real-world training experience is prohibitively expensive.Source: §3.2.2 · p. 32
• Reward shaping can cause specification gaming or local optima, while sparse success-only rewards avoid those pitfalls but typically slow learning.Dense rewards can guide long-horizon exploration, but designing them is error-prone and relies heavily on human expertise and intuition.Source: §3.2.2 · p. 32
• Behavioral Cloning can bypass simulator and reward-design requirements by reproducing demonstrations, but imitation performance can at most match the demonstrator.The text describes BC as increasingly competitive and practical, despite this inherent suboptimality.Source: §3.2.2 · p. 32

• BC casts control as imitation from previously collected expert demonstrations, avoiding online exploration and hand-crafted reward shaping while inheriting objectives and success criteria encoded in the data.This can reduce early-stage exploratory failures.Source: §4 · p. 33
• In its supervised-learning formulation, BC maps observations such as images and proprioception to expert actions, using reward-free trajectories of observation-action pairs.Demonstrations may have different lengths and multimodal strategies.Source: §4 · pp. 33–34
• Because demonstrations are sequential rather than i.i.d., shuffling can support Monte Carlo risk estimates, while DAgger is out of scope when only one offline dataset is available.The passage notes shuffled estimates may be less accurate.Source: §4 · p. 34
• BC limits exploration risks, avoids reward design and associated reward-hacking risks, and can encode terminal conditions, success detection, and resets; its performance is reported to scale with growing demonstration corpora.BC can reproduce behavior no better than the demonstrator, making scarce or suboptimal demonstrations a constraint.Source: §4 · p. 34
• Point-estimate policies can compound small prediction errors under covariate shift and average across equally viable demonstration modes, producing indecisive or unsafe commands.Generative modeling of the joint observation-action distribution is proposed to address poor multimodal fitting.Source: §4 · pp. 34–35, Figure 20 · p. 35

4.1 A (Concise) Introduction to Generative Models

• Generative models fit a parametric distribution to expert observation-action data so new samples resemble the dataset and observed regions receive high likelihood.Likelihood-based learning supplies a training objective for this distribution-matching goal.Source: §4.1 · p. 35

4.1.1 Variational Auto-Encoders

• VAEs model observation-action pairs through a latent variable and use an approximate posterior to make the variational lower-bound objective tractable.Maximizing the ELBO also maximizes data log-likelihood; the common formulation balances reconstruction with latent regularization.Source: §4.1.1 · pp. 36–37

4.1.2 Diffusion Models

• Diffusion models use a hierarchy of Markov-dependent latents, progressively corrupting data toward Gaussian noise and learning to reverse the corruption through denoising.The simplified training objective predicts total displacement for a randomly selected diffusion duration.Source: §4.1.2 · pp. 38, 41, Figure 24 · p. 40

4.1.3 Flow Matching

• Flow matching learns a vector field that transports samples along continuous paths from an easy-to-sample prior toward the data distribution.Optimal-transport-inspired paths tend to be straighter than diffusion paths and can reduce inference compute while retaining comparable results.Source: §4.1.3 · pp. 42–43

4.2 Action Chunking with Transformers

• ACT learns chunks of multiple actions from offline trajectories to mitigate compounding errors in behavior cloning.Its approach draws on how humans plan sequences of actions rather than single actions.Source: §4.2 · p. 43
• ALOHA pairs ACT’s manipulation work with a bimanual robot setup that costs as much as a mono-arm Franka and uses easy-to-source parts.The tutorial describes tasks including opening a lid, slotting a battery, and preparing tape.Source: §4.2 · p. 43
• ACT uses a transformer-based conditional VAE to model the policy distribution p(a|o), rather than the full joint distribution over observations and actions.The tutorial notes that conditional distributions can be intractable when represented through function approximators.Source: §4.2 · p. 44
• 1% vs. 44% success rate with action chunking; on human demonstrations, the standard supervised objective reduced success by 33.3% relative to the variational objective.The tutorial attributes the objective difference to the multimodal nature of human demonstration data.Source: §4.2 · p. 44
• At inference, ACT sets the latent style variable z to zero and feeds visual and proprioceptive observations to the decoder; its CVAE encoder is used only during training.The training encoder estimates the latent posterior from actions and proprioceptive states, while the decoder produces action chunks.Source: §4.2 · p. 44, Figure 28 · p. 45, Figure 29 · p. 45

4.2.1 Code Example: Training and Using ACT in Practice

• The code example maps dataset features into ACT inputs and outputs, creates a LeRobotDataset, and builds an offline-training optimizer.It configures ACT from the dataset’s features before training.Source: §4.2.1 · pp. 46–47

4.3 Diffusion Policy

• Diffusion Policy models p(a|o) and predicts multiple future actions, conditioning its noise regressor on a stack of previous observations.The tutorial describes this observation conditioning and action-chunk prediction as helping the policy commit to data modes at inference.Source: §4.3 · p. 49
• The convolution-based architecture embeds noisy action chunks and observations, then uses a U-Net conditioned at every layer to predict and iteratively remove noise.Figure 31 depicts observation-conditioned denoising of action chunks with as few as T = 10 steps.Source: §4.3 · p. 49, Figure 31 · p. 49
• High-frame-rate RGB observations attain success rates comparable to state-based policies, which are typically trained in simulation with privileged information.The tutorial connects this result to learning streamlined control policies directly from pixels.Source: §4.3 · p. 49
• Diffusion Policy reliably outperforms the considered baseline for all benchmark sizes considered.The tutorial also reports strong performance across simulated and real-world tasks.Source: §4.3 · pp. 49–50

4.3.1 Code Example: Training and Using Diffusion Policies in Practice

• The code example configures policy inputs and outputs from dataset features, creates a LeRobotDataset, and builds an offline-training optimizer.It then runs a training loop for the diffusion policy.Source: §4.3.1 · pp. 50–51

4.4 Optimized Inference

• Synchronous inference reduces average control-time computation by predicting only after a chunk is exhausted, but leaves the robot idle during inference.With g = 0, the average idle time is E[ℓS].Source: §4.4 · pp. 52–54
• Decoupling chunk prediction from execution lets the client consume existing actions while inference runs, and permits inference on a remote server without processing every timestep.The RobotClient aggregates overlapping old and incoming chunks when the new chunk arrives.Source: §4.4 · p. 53
• The latency analysis approximates expected round-trip latency by server inference latency when communication is symmetric and negligible relative to inference.At 30 fps, the stated control cycle is 33 ms.Source: §4.4 · p. 54
• For 0 < g < 1, inference overlaps action consumption, while g = 1 triggers inference every timestep and can be prohibitively expensive on limited hardware.Intermediate thresholds amortize computation and keep the queue from emptying; overlapping chunks buffer against modeling errors.Source: §4.4 · p. 54
• The threshold g trades idle periods at low values against higher compute costs near 1, and joint-space filtering drops near-duplicate observations before processing.If the action queue empties, the most recent observation is processed regardless of similarity.Source: §4.4 · pp. 54–55

4.4.1 Code Example: Using Async Inference

• The code example imports server and client components, configures the robot and cameras, then starts the client control loop.It also shows starting an action-receiver thread and optionally visualizing action-queue size.Source: §4.4.1 · pp. 55–56

5.1 Preliminaries: Models and Data

• Robotics data are embodiment- and task-specific, costly, and heterogeneous, making naive aggregation risky; Open-X nevertheless demonstrated positive transfer across tasks and embodiments.Open-X aggregated 60 datasets into 1.4M trajectories; DROID added 75k+ in-the-wild demonstrations, while LeRobot supports decentralized data collection.Source: §5.1 · pp. 58–59

5.2 VLAs

• Modern VLAs combine pretrained VLM backbones with action-specific components, using continuous action distributions and action chunking to support multimodal control and reduce error compounding.Separate perception and action components can support more efficient Mixture of Experts architectures; the paper links this recipe to few-shot adaptation and dexterous manipulation.Source: §5.2 · p. 60

5.2.1 VLMs for VLAs

• π0 routes vision and language through a pretrained VLM and robot state and action chunks through a separate action expert, with flow matching used to generate continuous actions.The two components communicate through self-attention while retaining disjoint weights; Driess et al. (2025) show that failing to insulate VLM knowledge from flow-matching gradients harms performance.Source: §5.2.1 · pp. 61–62
• Across benchmarks, π0 pretrained on the π dataset and fine-tuned on higher-quality demonstrations consistently outperforms πscratch, while retaining robustness and fluent multi-stage behavior.The paper attributes the advantage partly to large-scale demonstrations containing diverse information, including recovery from near-failure states that high-quality demonstrations may omit.Source: §5.2.1 · pp. 62–63

5.3.1 Code Example: Using π0

• The excerpt imports PI0Policy and SO100FollowerConfig, and sets preprocessor_overrides so the device_processor uses device "mps", identifying the selected policy, robot configuration, and processor setting.Source: §5.3.1 · p. 63

5.4 SmolVLA

• SmolVLA is an open-source effort that releases its model, data, and training recipes, targeting community-contributed datasets and accessible SO-100 and SO-101 arms.It pretrains on 450+ community datasets totaling 20k+ trajectories; task instructions are re-annotated with an off-the-shelf VLM, and camera viewpoints are standardized to top/wrist/side order.Source: §5.4 · pp. 64–65
• SmolVLA combines a pretrained VLM backbone with a dedicated action expert and flow matching, using about 450M parameters versus π0’s 3.3B.Its inputs include multi-view RGB images, a language instruction, projected sensorimotor state, and a noised action chunk; interleaved cross- and self-attention layers condition action tokens on visual, language, and proprioceptive features.Source: §5.4 · p. 65
• SmolVLA reduces inference compute by limiting visual tokens to 64 per frame and using only the first half of VLM decoder layers.The inference stack decouples action prediction from execution for responsiveness on modest hardware.Source: §5.4 · p. 65
• SmolVLA rivals π0 across real-world and simulated environments while running close to 40% faster and consuming 6x less memory.At test time, it forward-integrates flow over 10 steps.Source: §5.4 · p. 65

5.4.1 Code Example: Using SmolVLA

• The SmolVLA code example configures an SO100 follower and cameras, maps hardware observation and action features to policy inputs, then sends predicted actions to the robot.Camera keys and resolutions must match those used in training.Source: §1.3 · p. 7, §5.4.1 · p. 66

6 Conclusions

• Traditional dynamics-based control is brittle and engineering-intensive, motivating more flexible learning-based alternatives.Conventional explicit models also demand extensive human expertise, hindering robotics’ accessibility and scalability.Source: §6 · p. 67
• RL enables learning through interaction, but sample inefficiency, real-world safety concerns, and reward design complicate its use in robotics.HIL-SERL makes real-world RL more feasible by incorporating human guidance, previously collected data, and learned reward classifiers.Source: §6 · p. 67
• Imitation learning offers a way to learn safely from limited, reward-free expert demonstrations, while BC models learn complex multimodal behaviors.The tutorial traces this progression from single-task policies to language-conditioned VLA models that perform few- and zero-shot tasks across real-world settings.Source: §6 · p. 67
• The tutorial argues that open datasets, accessible architectures, and open-source software democratize access and support decentralized robotics research.It presents openness as a fundamental enabler of progress in the field.Source: §6 · p. 67
• The tutorial aims to equip researchers and practitioners with context and tools to explore open-source robot learning.It frames this as a journey from first principles to the state of the art.Source: §6 · p. 67