PaperlayerView exact revision on arXiv

Source-linked AI summary

Behavioral Cloning Mystery

Seohong Park, Sergey Levine

arXiv:2610.07056v1cs.ROcs.LG

TL;DR

Behavioral-cloning phenomena are challenging to study because real-world data and experiments are costly and difficult to control, while synthetic benchmarks often miss them. OCBench is a GPU-accelerated simulated benchmark whose controllable scripted policies mimic key structural properties of human demonstrations. OCBench reproduces many anecdotal behavioral-cloning phenomena in controlled settings and generates trajectories at up to 200K environment steps per second.

  • Problem

    Behavioral-cloning phenomena are challenging to study because real-world data and experiments are costly and difficult to control, while synthetic benchmarks often miss them.

    Source: §1 · p. 1

  • Method

    OCBench is a GPU-accelerated simulated benchmark whose controllable scripted policies mimic key structural properties of human demonstrations.

    Source: §1 · p. 1, §4.1 · p. 4

  • Results

    OCBench reproduces many anecdotal behavioral-cloning phenomena in controlled settings and generates trajectories at up to 200K environment steps per second.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    OCBench supports controlled scientific study of reported behavioral-cloning phenomena and their hypotheses using human-like scripted data.

    Source: §1 · p. 2

    OCBench’s policies remain scripted and do not perfectly mimic human behavior, so robotics research ultimately still requires real-world work with real data.

    Source: §6 · p. 10

Abstract

from arXiv · show

Behavioral cloning (BC), despite its simplicity, exhibits many counterintuitive phenomena in the real world. For example, the performance of BC often keeps increasing as the model overfits more to the dataset, and fully closed-loop policies often completely fail without action chunking. Unfortunately, properly studying these anecdotal phenomena ("behavioral cloning mysteries") is challenging: in the real world, datasets and experiments are costly and not fully controllable; in simulation with synthetic data, these phenomena are often not easily observed partly due to the discrepancy between scripted policies and human demonstrations. In this work, we propose OCBench, a robotic manipulation benchmark with controllable scripted policies that have similar properties to human demonstrations. We show that, by mimicking key properties of human demonstrations, OCBench reproduces many anecdotal BC-related phenomena in controlled settings. With its GPU-accelerated environments and scripted policies, we demonstrate how OCBench enables scientific studies of previously reported BC-related phenomena by analyzing and refuting various hypotheses. Project page: https://seohong.me/projects/ocbench

1 INTRODUCTION

Behavioral cloning exhibits counterintuitive phenomena, but costly, less-controllable real-world experiments and synthetic benchmarks that miss human-demonstration properties make them difficult to study. OCBench pairs human-like scripted data with controlled experiments to reproduce reported phenomena and analyze hypotheses.

1 INTRODUCTION

  • Real-world experiments are costly and difficult to control, while synthetic scripted-policy benchmarks often fail to reproduce real-world behavioral-cloning phenomena.The authors attribute this gap likely to differences between prior scripted policies and human demonstrations.Source: §1 · p. 1
  • OCBench is a suite of simulated tasks and datasets designed to share human demonstrations’ structural properties for controlled studies of realistic behavioral cloning and robotic control.The paper identifies narrow, non-Markovian, smoothly random trajectories as examples of those properties.Source: §1 · p. 1
  • On a single H100 GPU, OCBench collects up to 200K environment steps per second, enabling streaming-data training with continually generated data.The authors use this capability to study algorithm and design-choice limits under effectively unlimited data.Source: §1 · p. 2
  • OCBench reproduces four previously reported non-intuitive behavioral-cloning phenomena and provides analyses and hypotheses supported by controlled evidence.The authors present it as a way to make these phenomena scientifically studyable.Source: §1 · p. 2

2 RELATED WORK

Prior work studies behavioral-cloning methods and failure modes, but fixed human datasets and synthetic benchmarks leave differences in data scale and demonstration structure relevant to interpretation. OCBench instead uses controllable, human-like scripted policies and GPU-accelerated generation.

2 RELATED WORK

  • The paper focuses mainly on flow behavioral cloning, while noting that its studied phenomena may also apply to other behavioral-cloning techniques.The cited method families include MLPs, tokenized actions, energy-based models, diffusion, and flow matching.Source: §2 · p. 2
  • Earlier analyses often used only 50–300 human demonstrations per task, leaving unclear whether reported issues persist at the larger data scales used by recent robotic systems.OCBench uses GPU-accelerated scripted policies to generate effectively unlimited data while reproducing many real-world behavioral-cloning phenomena.Source: §2 · p. 2
  • Unlike MimicGen’s replay of segments from about 10 recorded human trajectories per task, OCBench uses fully scripted policies with controllable generation knobs.The knobs include motion speed, mistake probability, and coverage degree.Source: §2 · p. 2
  • OCBench’s scripted policies have more human-like action statistics than prior scripted or RL-trained policies, which tend to saturate action boundaries or have overly uniform distributions.The comparison draws on action distributions from OGBench, Robomimic MG, OCBench, and human demonstrations.Source: Figure 1 · p. 3
  • OCBench distinguishes itself from prior scripted or RL-trained benchmark policies by aiming to reproduce real-world phenomena that those policies do not readily reveal.The authors identify OGBench as OCBench’s closest predecessor.Source: §2 · p. 3

3 MOTIVATION

The paper frames four counterintuitive behavioral-cloning findings as mysteries and argues that limited control over human-generated data and synthetic-policy differences complicate their study. It hypothesizes that demonstration structure contributes to the discrepancy.

3 MOTIVATION

  • Behavioral-cloning performance can increase with greater overfitting, and smaller datasets can sometimes outperform larger ones (Mandlekar et al., 2021; Allshire et al., 2026; Codevilla et al., 2019; Bai et al., 2025).The passage presents these as reported phenomena, not conclusions established in this section.Source: §3 · p. 3
  • Partially open-loop policies with action chunking can outperform closed-loop policies even without inference delay, and history conditioning does not necessarily resolve the gap (Zhao et al., 2023; Lazzati et al., 2026; Zeng et al., 2026; de Haan et al., 2019; Wen et al., 2020; Sobol Mark et al., 2026).Here, “open-loop” means partially open-loop control with action chunking (Zhao et al., 2023).Source: §3 · p. 3
  • Behavioral cloning on human-collected data can require much larger networks than cloning synthetic data, including on state-based tasks (Pan et al., 2026).Source: §3 · p. 3
  • Behavioral-cloning performance can depend heavily on input features, and scaling data does not necessarily reduce that dependence (de Haan et al., 2019; Mandlekar et al., 2021).Source: §3 · p. 3
  • Real-world human-generated data are difficult to control, while existing scripted or RL-trained benchmarks do not consistently reproduce reported phenomena.The paper gives a less pronounced open-loop versus closed-loop gap and lower policy-size requirements in some popular synthetic benchmarks as examples.Source: §3 · p. 4
  • The authors hypothesize that prior synthetic policies differ from human demonstrations in temporal correlation, multimodality, and action statistics, including boundary-biased or overly uniform actions.They propose that OCBench aims to bridge this gap.Source: §3 · p. 4

4 OCBENCH: A CONTROLLABLE ROBOTIC MANIPULATION BENCHMARK

OCBench’s controllable scripted policies aim to reproduce the structure of human demonstrations in a GPU-accelerated manipulation benchmark. The authors report that these policies reproduce counterintuitive behavioral-cloning phenomena that prior scripted benchmarks do not readily reveal.

4 OCBENCH: A CONTROLLABLE ROBOTIC MANIPULATION BENCHMARK

  • OCBench’s goal is to support controlled studies of behavioral-cloning and potentially reinforcement-learning phenomena using human-demonstration-like scripted data and GPU-accelerated MuJoCo.Source: §4 · p. 4

4.1 SCRIPTED POLICIES AND DATASETS

  • The scripted policies target narrow trajectory distributions, strong non-Markovian temporal correlation, and semantic diversity in poses and strategies.They use randomized piecewise C1-continuous Hermite splines, varying waypoints, grasping, motion, and task strategies, and occasionally make and recover from mistakes.Source: §4.1 · p. 4
  • OCBench’s policies reproduce many counterintuitive phenomena observed with human-data behavioral cloning, unlike previous scripted benchmarks, and have more human-like action statistics.The paper reports these outcomes for policies designed to mimic human-demonstration properties.Source: §4.1 · p. 4

4.2 TASKS

  • OCBench provides 28 tasks across five environment types: block, chamber, switch, hanoi, and bowling.The tasks present diverse manipulation challenges.Source: §4.2 · p. 4, Figure 2 · p. 5
  • The benchmark provides CPU and MJWarp versions with logically identical dynamics and data-generation strategies, plus lite versions of most environments for lighter experimental iterations.The environments operate at 50Hz with a 7-dimensional joint-delta action space; multi-view pixel observations are also provided.Source: §4.2 · pp. 4–5

5 STUDYING BEHAVIORAL CLONING IN OCBENCH

OCBench provides controlled, GPU-accelerated studies of four counterintuitive behavioral-cloning phenomena using scripted policies designed to mimic human demonstrations. Across the studies, it reproduces these phenomena and tests possible explanations, without fully resolving them.

5.1 MYSTERY 1: OVERFITTING IS NOT NECESSARILY BAD

  • Performance can keep rising as validation flow loss grows, while larger same-distribution datasets can significantly hurt performance, even with independent and identically distributed samples.The size-reversal persists with 540M-sized models, so limited model capacity alone is insufficient to explain it; action MSE tracks performance better than flow loss but does not fully explain the results.Source: §5.1 · p. 6

5.2 MYSTERY 2: OPEN-LOOP IS OFTEN BETTER THAN CLOSED-LOOP

  • With streaming data, open-loop action chunking succeeds where closed-loop policies nearly fail, and Markovianizing demonstrations reduces—but does not eliminate—the performance gap.Randomly delayed closed-loop control performs strongly but still trails open-loop control, suggesting compounding errors alone may not explain the gap.Source: §5.2 · pp. 7–8

5.2.1 ASIDE: WHAT ABOUT HISTORY CONDITIONING?

  • Under streaming data, history-conditioned policies can perform worse despite lower flow loss and action MSE, so better dataset fit does not guarantee better test performance.The authors leave the explanation unresolved, noting possible causal confusion or greater susceptibility to test-time distribution shift.Source: §5.2.1 · p. 8

5.3 MYSTERY 3: POLICIES SHOULD BE (VERY) LARGE

  • Performance on two basic state-based tasks keeps improving up to [8192] × 8 residual MLPs with about 0.5 billion parameters; FSQ policies favor smaller models than flow-based policies, but still benefit from scaling.The FSQ comparison indicates that flow matching alone does not explain the need for large-capacity networks.Source: §5.3 · pp. 8–9, Figure 10 · p. 9, Figure 11 · p. 9

5.4 MYSTERY 4: FEATURES MATTER EVEN WITH ENOUGH DATA

  • With unlimited streaming data, input scaling changes task performance substantially despite identical information and nearly indistinguishable training and validation metrics.On test-time states, policies with different features behave differently; test-time MSE correlates better with performance than training or validation MSE, though it ignores multimodality.Source: Figure 12 · p. 9, §5.4 · p. 10

6 CLOSING REMARKS

OCBench reproduces diverse real-world behavioral-cloning phenomena in controlled studies, while remaining a complement to real-world research. Its data and scripts are also designed to support future reinforcement-learning studies.

6 CLOSING REMARKS

  • Test-time MSE correlates with performance better than validation MSE.Figure 13 presents this comparison.Source: Figure 13 · p. 10
  • OCBench’s controllable, efficient data-generation scripts experimentally reproduce diverse real-world behavioral-cloning phenomena and support their scientific study.The authors describe OCBench as a fast, convenient playground that complements real-world research.Source: §6 · p. 10
  • OCBench’s scripted policies and datasets are designed to support future reinforcement-learning studies of success rates, throughput, and hierarchical control.Policy speeds vary by around 2×, and oracle subtask boundaries and labels are provided.Source: §6 · p. 10
  • Scripted policies do not perfectly mimic human behavior, so the authors say robotics research should ultimately use real-world data.They position OCBench as a complement for fast, controlled studies.Source: §6 · p. 10

A TASKS AND DATASETS

OCBench contains 28 robotic manipulation tasks across five environment types, with task variants, lite and pixel-based versions, and datasets carrying auxiliary information. Its metrics and task designs cover success, throughput, and varied manipulation challenges.

A TASKS AND DATASETS

  • OCBench provides 28 tasks across block, chamber, switch, hanoi, and bowling environments.Block, chamber, and switch are modified from OGBench environments; OCBench also includes standard robotic tasks beyond goal-conditioned ones.Source: A TASKS AND DATASETS · p. 16
  • Block tasks span pick-and-place, stacking, and grid arrangement; chamber tasks combine object manipulation with randomized layouts, subtask order, and occasional mistakes.Chamber demonstrations can extend to 5500 steps.Source: A TASKS AND DATASETS · p. 16
  • Switch tests combinatorial button puzzles and suboptimal demonstrations, hanoi requires precise disk moves, and bowling tests pushing under unpredictable dynamics.Hanoi tasks extend up to 7000 environment steps; bowling rewards pins knocked down.Source: A TASKS AND DATASETS · p. 16
  • Lite versions, except for bowling, use shorter horizons and simpler policies; they run at 10Hz and move about 2× faster than original tasks.Their horizons are approximately 10× shorter, and smaller networks can be used.Source: A TASKS AND DATASETS · pp. 16–17
  • Pixel-based tasks use front, side, and wrist images of 224 × 224 × 3, recorded every 25 steps to reduce dataset size and speed data generation.Datasets are supplied only for selected visual tasks due to storage constraints; data can also be generated with the scripts.Source: A TASKS AND DATASETS · p. 17, Figure 14 · p. 17
  • Most tasks use binary sparse rewards and are evaluated by success rate and throughput; bowling uses normalized average return instead of success rate in the tables.For bowling, normalized average return is average return multiplied by 100; throughput uses mean episode length across all episodes, including failures.Source: A TASKS AND DATASETS · p. 17
  • Datasets include reconstructible full simulation states and per-subtask segments, target information, success indicators, and high-level mistake indicators.The authors anticipate these data will support reinforcement-learning, hierarchical-control, and fine-grained analyses.Source: A TASKS AND DATASETS · p. 17

B HOW IS OCBENCH DIFFERENT FROM PREVIOUS SYNTHETIC BENCHMARKS?

OCBench differs from previous synthetic benchmarks through more human-like scripted policies, GPU acceleration, and support for standard robotic tasks. Its trajectories also reproduce several behavioral-cloning patterns that are less evident in earlier synthetic benchmarks.

B HOW IS OCBENCH DIFFERENT FROM PREVIOUS SYNTHETIC BENCHMARKS?

  • The closed-loop versus action-chunked performance gap is less pronounced in OGBench and Robomimic machine-generated data than in OCBench or human-generated data.Previous synthetic policies typically produce trajectories with less temporal correlation than human demonstrations, which could explain the smaller gap.Source: B HOW IS OCBENCH DIFFERENT FROM… · pp. 17–18, Figure 15 · p. 18
  • Networks of size [256] × 8 saturate performance on previous synthetic benchmarks, while OCBench benefits from models of up to hundreds of millions of parameters on similarly complex tasks.The authors associate this scaling pattern with OCBench trajectories being narrower and more temporally correlated; Figure 16 compares the model-size results.Source: B HOW IS OCBENCH DIFFERENT FROM… · p. 18, Figure 16 · p. 18
  • OCBench action distributions more closely resemble human demonstrations than the often bimodal or overly uniform distributions of previous synthetic datasets.The absolute action values may not be directly comparable because robot joint ranges differ.Source: B HOW IS OCBENCH DIFFERENT FROM… · p. 18
  • OCBench’s scripted policies are designed to mimic human demonstrations, support GPU acceleration, and cover standard robotic tasks beyond goal-conditioned settings.Table 1 summarizes these differences from OGBench.Source: B HOW IS OCBENCH DIFFERENT FROM… · p. 18, Table 1 · p. 19

C ADDITIONAL RESULTS

Additional results indicate that action MSE does not fully account for the dataset size-reversal phenomenon, while pixel-based OCBench tasks reproduce three BC-related phenomena. The paper also reports preliminary results for three BC algorithms.

C ADDITIONAL RESULTS

  • Action MSE does not fully explain the dataset size-reversal phenomenon.Figure 17 compares performance, training and validation flow losses, and training and validation action MSE across dataset sizes.Source: Figure 17 · p. 19, C ADDITIONAL RESULTS · p. 19
  • Pixel-based tasks reproduce three phenomena: overfitting need not hurt performance, open-loop control improves performance over closed-loop control, and test-time MSE can better explain performance.For two features, states versus pixels, test-time MSE can trend oppositely to validation MSE.Source: C ADDITIONAL RESULTS · pp. 19–20, Figure 18 · p. 19
  • The paper provides preliminary benchmark results for BC, flow BC, and FSQ BC, averaged over two seeds with standard deviations reported.The results are presented in Tables 6 and 7 as a reference; the paper’s focus is analysis rather than algorithm comparison.Source: C ADDITIONAL RESULTS · p. 20

D IMPLEMENTATION DETAILS

The implementation section provides details and reproducibility materials for the benchmark tables.

D IMPLEMENTATION DETAILS

  • The paper provides full implementation details for its experiments.It also provides code, datasets, and exact command-line arguments to reproduce Tables 6 and 7.Source: D IMPLEMENTATION DETAILS · p. 20
  • Code and datasets are available for reproducing the benchmark tables.The paper also provides exact command-line arguments.Source: D IMPLEMENTATION DETAILS · p. 20
  • Exact command-line arguments are provided for reproducing Tables 6 and 7.The tables report benchmark results.Source: D IMPLEMENTATION DETAILS · p. 20

D.1 METHODS

The methods section describes vanilla BC, flow BC, and FSQ BC, alongside training defaults and a visual-observation approach.

D.1 METHODS

  • By default, agents train on successful episodes except for bowling, use length-25 action chunks or length-3 chunks for lite tasks, and use residual MLPs.Actions are percentile-normalized per dimension except for the gripper, which is clipped directly; streaming runs collect 25K episodes every 100K steps and discard old data.Source: §D.1 · p. 20
  • Vanilla BC trains a deterministic policy with standard MSE loss.Source: §D.1 · p. 20
  • Flow BC trains a velocity field with flow matching and uses Beta(1, 1.5) flow times, which improves training speed in OCBench but makes no clear difference in OGBench.This differs from previous OGBench flow-based works, which use a uniform time distribution.Source: §D.1 · p. 20
  • FSQ BC jointly trains an action-chunk FSQ autoencoder and an autoregressive network that predicts its tokens from state.By default, each chunk is encoded into four tokens with quantization levels [8, 5, 5, 5].Source: §D.1 · p. 20
  • For visual observations, the method encodes three images with a shared ConvNeXt V2-N encoder, concatenates their representations, and feeds them to a residual MLP.Random-crop image augmentation is applied by default during training.Source: §D.1 · p. 20

D.2 HYPERPARAMETERS

The section points to default hyperparameters and task specifications, and identifies the tables reporting benchmark success rates and throughput.

D.2 HYPERPARAMETERS

Related papers

Loading 2610.07056v1…