PaperlayerView exact revision on arXiv

Source-linked AI summary

Training-free Behavior Cloning

Maximilian Adang, Timothy Chen, Lars Osterberg, Aiden Swann, Mac Schwager

arXiv:2609.30134v1cs.RO

TL;DR

Neural behavior cloning makes actions difficult to trace and policy updates costly, while retrieval policies face mismatch between live histories and demonstrations. BPC combines action-aware retrieval, history reconstruction for action continuation, and residual correction; reported results are competitive across simulation and hardware, with fitting reduced from hours to seconds and control at 75 Hz on a Jetson Orin Nano.

  • Problem

    Neural policies make individual actions difficult to attribute and updates costly, while retrieval policies struggle when live observation–action histories differ from recorded trajectories.

    Source: §I · pp. 1–2

  • Method

    BPC retrieves demonstration windows, reconstructs the live history to form an action-continuation prior, and applies a learned residual correction.

    Source: §I · p. 2

  • Results

    BPC is reported as competitive across simulation and hardware, with fitting reduced from hours to seconds and control at 75 Hz on a Jetson Orin Nano.

    Source: §I · p. 2

  • Takeaways & Limitations

    Retrieved windows and coefficients estimate task progress, while retaining demonstrations makes behavior traceable and the demonstration bank a point of intervention.

    Source: §I · pp. 1–2

    BPC depends on demonstration coverage and informative histories, and does not guarantee closed-loop stability, safety, or task completion after residual correction.

    Source: §VI · p. 7

Abstract

from arXiv · show

Neural behavior cloning compresses demonstrations into large models, making individual actions difficult to trace and policy updates costly. Retrieval policies retain access to demonstrations but struggle with mismatch between recorded and live behavior. We introduce Behavior Predictive Control (BPC), which synthesizes policies without end-to-end policy training by combining an action-aware retrieval metric, a Hankel-based action-continuation prior, and a closed-form one-step residual correction. Inspired by behavioral systems theory, BPC predicts future actions by blending stored observation-action data that best reconstructs the recent runtime observation--action history. Across simulated benchmarks and real-robot deployments, BPC is competitive with learned policies such as $\pi_{0.5}$ (surpassing it in some cases), while reducing policy fitting from hours to seconds on consumer GPUs and supporting closed-loop control upwards of 75 Hz on a Jetson Orin Nano. The retrieved demonstration windows and their coefficients also provide an intrinsic estimate of task progress. Retaining demonstrations within the deployed policy makes its predictions traceable to supporting trajectories and enables behavior revision through the demonstration bank.

I. INTRODUCTION

BPC retains demonstrations at deployment and uses retrieved trajectories to continue behavior from the live observation–action history, addressing the mismatch that limits direct retrieval. The paper positions this approach alongside behavioral-systems methods, neural behavior cloning, and retrieval-based imitation.

I. INTRODUCTION

  • BPC addresses query–demonstration mismatch by reconstructing the live history over retrieved trajectories and carrying the fitted coefficients into their future actions.Retrieval policies keep demonstrations explicit, exposing supporting data and making the bank a point of intervention.Source: §I · pp. 1–2
  • BPC combines learned retrieval, regularized history reconstruction for an action-continuation prior, and a closed-form residual correction.Its residual is fitted on the same retrieved support to address nonlinear or higher-order effects.Source: §I · p. 2
  • Across simulation and hardware, BPC is reported as competitive across tasks and embodiments, with fitting reduced from hours to seconds and control at 75 Hz on a Jetson Orin Nano.Retrieved windows and coefficients also provide an intrinsic task-progress estimate.Source: §I · p. 2

II. RELATED WORK

  • Trajectory completion and DeePC motivate using recorded trajectories to continue behavior, but BPC adapts that view to expert actions without assuming demonstrations satisfy the fundamental lemma’s conditions.The related-work discussion grounds this approach in behavioral-systems theory and trajectory-template methods.Source: §II · p. 2
  • Local and latent DeePC methods use compression, selection, or nonlinear representations; BPC instead fits a history distance using demonstrated continuations to select windows.The paper describes BPC as flexible from pretrained encoders to raw pixels.Source: §II · p. 2
  • Compared with neural behavior cloning and other retrieval-based imitation methods, BPC retains demonstration access while computing its action prior from local trajectory data.The related-work discussion contrasts neural action predictors with retrieval approaches including VINN, Behavior Retrieval, and DARP.Source: §II · p. 2

III. REGULARIZED LOCAL ACTION CONTINUATION

The section frames behavior cloning as predicting the expert’s next action from a live observation–action history using finite demonstration trajectories. Its trajectory-template setup divides windows into history and continuation so the observed past can inform the next action.

III. REGULARIZED LOCAL ACTION CONTINUATION

  • The behavior-cloning objective is to predict the expert action following the observed history, rather than future observations or a reward-guided reference trajectory.The demonstration library supplies recorded behaviors, and the live history is the query.Source: §III · pp. 2–3
  • Each admissible demonstration window is split into an observed history and a future continuation, with concatenated Hankel blocks kept within demonstration boundaries.The continuation supplies the expert actions to be inferred from the query history.Source: §III · p. 3
  • The setup assumes a compatible history identifies the expert’s next action, but explicitly does not assume the library covers every history the learner may visit.At deployment, future query data, an expert-action oracle, reward, and reference trajectory are unavailable.Source: §III · p. 3

A. Policy construction

Policy construction selects local trajectory templates and chooses coefficients that determine both how their histories reconstruct the query and which continuation is predicted. These choices encompass single-neighbor selection, averaging, and fitted local continuation.

A. Policy construction

  • The selected templates provide local history and continuation banks, with the first future action forming the one-step action bank.The banks support reconstructing the query history and predicting the continuation with shared coefficients.Source: A. Policy construction · p. 3
  • Choosing coefficients spans selecting one demonstration, averaging several, or fitting a local continuation.The discussion notes that affine continuation constraints may perform better than nearest-neighbor counterparts.Source: A. Policy construction · p. 3

B. Local approximation with signed coefficients

The section relates local continuation error to how well retrieved histories reconstruct the query and to the local behavior’s approximation properties. It uses this analysis to motivate action-aware support selection and signed affine coefficients, while noting their trade-off between reconstruction and coefficient amplification.

B. Local approximation with signed coefficients

  • The local continuation error proposition assumes a continuously differentiable policy with a Lipschitz Jacobian over a convex neighborhood containing the query and retrieved histories.The bound is stated for any coefficient vector.Source: B. Local approximation with signed… · p. 3
  • The error terms identify history mismatch, affine mass inconsistency, and curvature as distinct sources of next-action error.Some terms collapse under specific constraints on the coefficients.Source: B. Local approximation with signed… · p. 3
  • Because action sensitivity determines which history differences affect action error, retrieval should favor histories whose variations can reconstruct the query along action-relevant directions.Locality limits curvature cost, while redundant or distant histories can require large signed coefficients and amplify Taylor remainders.Source: B. Local approximation with signed… · pp. 3–4
  • Online BPC retrieves candidate windows, fits coefficients to reconstruct history, pools candidate features for a residual correction, and adds that correction to the action prior.The procedure first fits the retriever and residual offline, then performs retrieval, continuation, and correction at each time step.Source: Figure 2 · p. 4, B. Local approximation with signed… · p. 4
  • Affine continuation can reduce history mismatch, including for queries outside the retrieved convex hull, whereas nonnegative kNN averaging cannot reconstruct such queries.Affine continuation trades that flexibility for potentially greater coefficient amplification, so the support must be both local and sufficiently varied.Source: B. Local approximation with signed… · p. 4

IV. EXTENSION TO NONLINEAR BEHAVIORS

BPC extends local action continuation to nonlinear behaviors with future-informed retrieval and a nonlinear residual fitted on the same support. Retrieval offers action ridge and LDA pipelines that score candidates in a transformed history-feature space.

IV. EXTENSION TO NONLINEAR BEHAVIORS

  • A shared-support nonlinear residual corrects errors left by future-informed retrieval and local action continuation.The correction uses the same retrieved demonstrations and their differences from the current observation.Source: §IV · p. 4

A. Future-informed retrieval

  • Retrieval compares the live history with candidate histories in a transformed feature space, using action ridge or LDA pipelines.The feature map may be nonlinear, and either retrieval pipeline can be used depending on the data.Source: A. Future-informed retrieval · p. 4
  • Action ridge fits histories to future-action banks, then ranks candidates by predicted continuation similarity rather than predicting the action itself.Its linear predictor is used only for retrieval.Source: A. Future-informed retrieval · pp. 4–5
  • LDA weights histories according to similarity between their demonstrated futures, then averages history features to represent each candidate.The method uses random Fourier features for the history representation.Source: A. Future-informed retrieval · p. 5
  • The LDA pipeline regularizes feature covariance and offers a choice between fixed-size nearest-candidate retrieval and query-dependent sparsemax selection.Action ridge suits futures captured by a linear map of history; LDA suits similar futures whose histories vary substantially.Source: A. Future-informed retrieval · p. 5

B. Composing Retrieval Distributions

Retrieval distances can be interpreted as candidate probabilities and combined with additional task-relevant factors. A stateful task-progress prior reranks candidates before selection.

B. Composing Retrieval Distributions

  • Retrieval distances can define marginal candidate distributions that combine multiplicatively with other modality-specific factors.For non-Markovian tasks, a task-progress factor can rerank candidates using normalized demonstration progress and a stateful estimate bootstrapped from prior supports.Source: B. Composing Retrieval Distributions · p. 5
  • The task-progress prior uses a Laplace distribution around estimated progress, then selection proceeds on the history-aware reranked posterior.The estimate is the g-weighted mean progress of the previous step’s supports.Source: B. Composing Retrieval Distributions · p. 5
  • The progress estimate carries information forward from the previous step rather than relying only on the current observation.Candidate progress is normalized within each demonstration.Source: B. Composing Retrieval Distributions · p. 5

C. Shared-support nonlinear residual

The shared-support residual corrects action-continuation errors that can remain even when history reconstruction is accurate. BPC combines this correction with retrieval and local continuation.

C. Shared-support nonlinear residual

  • The residual uses the same retrieved candidates to correct action errors that remain after continuation, including when history fit is accurate.BPC’s three components are action-aware retrieval, local action continuation, and learned residual correction.Source: C. Shared-support nonlinear residual · p. 5
  • Random Fourier features are pooled uniformly across retrieved candidates and passed to a linear correction model.The correction model is W^T times the pooled feature.Source: C. Shared-support nonlinear residual · p. 5
  • Correction fitting excludes candidates from the same demonstration, measuring reconstruction without highly correlated query examples.The fitting procedure uses eligible history-action pairs and computes pooled features on the filtered support.Source: C. Shared-support nonlinear residual · p. 5

A. Tabletop Manipulation

On tabletop tasks, BPC substantially outperformed π0.5 on Switch while both policies had high reported success on Coffee and Drawer; in drone navigation, Figure-eight failures were attributed to perceptual aliasing. The section also reports competitive simulation results and rapid fitting.

A. Tabletop Manipulation

  • The xArm6 tabletop suite tested coffee-bag placement, opening and closing a drawer around sponge placement, and placing a bag on a left or right plate.The tasks used wrist and external cameras and proprioception.Source: A. Tabletop Manipulation · p. 6
  • BPC fit in 30–120 seconds on an RTX 4090, compared with 20 hours for π0.5; 16 × 16 raw pixels were sufficient for BPC to perform well.The xArm6 suite used visuomotor inputs; hardware manipulation used action ridge retrieval, while subsequent results used LDA.Source: A. Tabletop Manipulation · p. 6
  • 34 versus 8 successes out of 40 trials on Switch favored BPC over π0.5; Coffee and Drawer results were 36 and 39, and 38 and 40 successes out of 40, respectively.The authors attribute π0.5’s Switch degradation to its lack of memory about which plate the bag came from.Source: A. Tabletop Manipulation · p. 6
  • On simulated MimicGen tasks, BPC with R3M was competitive with state-of-the-art learned policies and superior to other retrieval policies.Fitting 1,000 demonstrations took 1–5 minutes, compared with hours for SDP and π0.5; Table III reports success rates over 300 rollouts per task.Source: A. Tabletop Manipulation · p. 6, Table III · p. 6

B. Drone Navigation

  • With 3–4 demonstrations per task, BPC’s test-time mean spatial distance from the drone training demonstrations was approximately 11 cm.The raw-pixel tasks were Loop and Figure-eight; Fig. 3 shows demonstrations, flown paths, predicted references, and the preceding two-second history, with demonstration segments colored by normalized coefficient magnitude.Source: B. Drone Navigation · p. 6, Figure 3 · p. 7
  • Figure-eight had two failures in ten trials, both attributed to perceptual aliasing at visually overlapping parts of the route.One failure repeated a trajectory section; the other entered the lower gate instead of the upper gate at the end.Source: B. Drone Navigation · p. 6

C. Dexterous Manipulation

BPC achieves competitive dexterous-manipulation results and fast computation, while its limitations include dependence on demonstration coverage and unresolved scaling to multi-task, much larger banks.

D. Ablations

  • Ablations identify all three pipeline components as synergistic and important, with correction having the largest impact, followed by action continuation and retrieval.The comparison removes history-conditioned continuation, nonlinear correction, or learned retrieval from the MimicGen visuomotor policy.Source: D. Ablations · p. 7, Table VI · p. 7

C. Dexterous Manipulation

  • Using 10 Adroit and 100 DexArt demonstrations per task, BPC is competitive with SDP-3D and has better success rates than DARP with the same inputs.Success rates are averaged over 120 rollouts; Table V reports the dexterous manipulation suites.Source: Table V · p. 7, C. Dexterous Manipulation · p. 7

E. Computation Time

  • On an 8 GB Jetson Orin Nano, BPC fits in under 3 minutes and runs onboard at 75 Hz.At massive dataset scales, fitting takes 3 minutes; on an RTX 4090, deployment reaches 100 Hz.Source: E. Computation Time · p. 7

VI. LIMITATIONS

  • BPC depends on adequate demonstration coverage and informative observation histories; behavior far from demonstrated states may require recovery actions absent from the bank.The analysis does not guarantee closed-loop stability, safety, or task completion after residual correction.Source: §VI · p. 7
  • BPC still requires retrieval and residual fitting, while demonstration storage and retrieval costs grow with dataset size; scaling to multi-task deployments remains future work.The paper describes the paradigm as compelling for large single-task datasets.Source: §VI · p. 7

VII. CONCLUSION

  • Keeping demonstrations in the deployed policy makes action provenance accessible, enables behavior revision, and provides an intrinsic estimate of task progress.The conclusion also identifies these properties as synergistic with agentic loops for retrieval, feedback, revision, and redeployment.Source: Figure 5 · p. 8

APPENDIX SIMULATION IMPLEMENTATION AND HYPERPARAMETERS

The appendix documents simulation-run settings, temporal control windows, observation encoders, and retrieval and residual fitting configurations.

APPENDIX SIMULATION IMPLEMENTATION AND HYPERPARAMETERS

  • The appendix specifies simulation configurations for Tables III and V and the tabletop ablations in Table VI, not hardware experiments or later task-specific settings.The reported benchmark configurations should not be read as covering hardware runs.Source: APPENDIX SIMULATION IMPLEMENTATION AND… · p. 10

A. Temporal Windows and Control Protocol

  • Simulation windows use a 10-step continuation and a 10-step history, except Square uses a 5-step history; the controller executes one action before updating history and predicting again.Each window spans 20 steps, or 15 for Square; the continuation is not executed as an open-loop chunk.Source: A. Temporal Windows and Control Protocol · p. 10, Table VII · p. 10

B. Observation Encoders

  • MimicGen observations combine two frozen R3M image embeddings with proprioception, while the policy receives no privileged simulator object-state vector.Each image embedding is 512D and proprioception is 9D, yielding 1,033 observation components; actions have 7 components.Source: B. Observation Encoders · p. 10
  • Dexterous-task BPC uses a PointNet point-cloud encoder trained per task with an auxiliary behavior-cloning objective, then frozen for retrieval and residual fitting.The reported configuration uses 64D point-cloud embeddings; frozen pretrained PointNet latents achieve slightly lower success rates on these tasks.Source: B. Observation Encoders · p. 10

C. Retrieval and Residual Hyperparameters

  • Retrieval and residual correction use distinct 16,384-feature random maps; retrieval reduces to rank 70 and fits on up to 8,192 anchors.The archived retrieval fit uses entropy-teacher Fisher-LDA; the affine readout uses an exact sum-to-one solver with no added ℓ1 penalty.Source: C. Retrieval and Residual… · p. 10

Related papers

Loading 2609.30134v1…