PaperlayerView exact revision on arXiv

Source-linked AI summary

DetAug: Obstacle-Blind Trajectory Augmentation for Zero-shot Obstacle Avoidance

Reece O'Mahoney, Moritz Zoellner, Ioannis Havoutis

arXiv:2609.18395v1cs.RO

TL;DR

Free-space manipulation demonstrations leave policies facing obstacles absent from training, while steering is limited by existing behaviors and synthetic augmentation requires obstacle geometry. DetAug adds labeled transit detours and selects among candidate trajectories; it reports stronger obstacle-avoidance results in simulation and on hardware.

  • Problem

    Free-space demonstrations leave a gap for obstacle deployment: steering is limited by prior behaviors, while synthetic augmentation requires obstacle geometry during training.

    Source: §I · p. 1

  • Method

    DetAug replaces transit segments with parameterized arcs while preserving grasp and release, conditions the policy on arc labels, and selects low-collision trajectories at inference.

    Source: §I · p. 2

  • Results

    SafeLIBERO collision-free success was more than 20pp above the next best method, with DetAug also outperforming inference-time methods on real hardware tasks requiring large detours.

    Source: §I · p. 2, E. Hardware Experiments · p. 6

  • Takeaways & Limitations

    DetAug matched or exceeded an obstacle-conditioned baseline on real hardware without seeing obstacles during training.

    Source: §V · p. 6

    With full fine-tuning, point-cloud conditioning can match or outperform DetAug, but billion-parameter VLA training may be prohibitively expensive and take many days.

    Source: B. Adapting a Pre-Trained Policy · p. 5

Abstract

from arXiv · show

Policies for robotic manipulation are produced by training on large teleoperated datasets. These datasets typically consist of free-space trajectories, making them difficult to transfer to test-time environments with obstacles. Previous methods for closing this gap have largely fallen into two groups. Dataset augmentation addresses it at training time but needs obstacle geometry in advance, whereas steering an existing checkpoint at inference time avoids that requirement but is limited in flexibility. Our method draws from both areas without inheriting either drawback. DetAug applies an obstacle-blind augmentation scheme to the transit phases of a free-space dataset, leaving object interactions untouched, and records the augmentation parameters as an explicit conditioning label. At inference it samples a batch of labels and executes the trajectory with the lowest collision cost. On the SafeLIBERO benchmark DetAug achieves a collision-free success rate more than 20pp above the next best method, and selecting over the label space outperforms guidance on the same policy by 26pp. On real hardware, inference-time steering methods collapse on tasks requiring large detours, while DetAug matches or exceeds an obstacle-conditioned baseline without ever seeing obstacles in training.

I. INTRODUCTION

Free-space manipulation policies face test-time obstacles, while prior approaches either need obstacle geometry during training or are limited by the behaviors already present in a policy. DetAug augments transit motions with labeled detours, preserves object interactions, and selects among labeled trajectories at inference.

I. INTRODUCTION

  • Free-space demonstrations often leave policies facing deployment obstacles absent from their datasets.Inference-time steering is limited by the prior’s existing behaviors, while synthetic augmentation requires obstacle geometry in advance.Source: §I · p. 1
  • DetAug changes only transit phases, preserving grasp and release windows while replacing motion segments with parameterized Cartesian arcs.The arc parameters become conditioning labels; inference samples label-conditioned trajectories and scores them for collisions at each replan.Source: §I · pp. 1–2, Figure 2 · p. 2
  • SafeLIBERO collision-free success was more than 20pp above the next best method, and label-space selection beat guidance on the same policy by 26pp.With a frozen backbone and 3M-parameter adapter, DetAug reached 35% success versus 2% for obstacle conditioning.Source: §I · p. 2
  • On tasks requiring large deviations, DetAug completes tasks better than inference-time methods in simulation and on real hardware.Those methods may avoid obstacles yet fail to complete the task.Source: §I · p. 2

A. Inference-Time Steering of Frozen Policies

Inference-time steering modifies a frozen policy, whereas train-time synthetic augmentation can introduce new behaviors but ties avoidance to sampled obstacle distributions. DetAug instead conditions on a low-dimensional arc label rather than scene geometry.

A. Inference-Time Steering of Frozen Policies

  • Frozen-policy steering can only reweight, deflect, or select behaviors already represented in the prior.DetAug diversifies with an explicit augmentation label rather than relying on collision guidance or searching the prior’s own modes.Source: A. Inference-Time Steering of Frozen… · p. 2

B. Train-Time Augmentation with Synthetic Obstacles

  • Obstacle-conditioned synthetic augmentation creates new behaviors, but its avoidance is fixed by the sampled obstacle distribution and dense geometry is harder to add to pretrained models.DetAug uses a low-dimensional arc descriptor that can be injected through a single zero-initialized modulation pathway.Source: B. Train-Time Augmentation with… · p. 3

C. Trajectory Stitching and Splitting

DetAug moves obstacle avoidance from inference-time plan splitting into data augmentation, then ranks generated trajectories using scene geometry. Its overview describes training on labeled arcs and selecting among sampled label-conditioned plans.

C. Trajectory Stitching and Splitting

  • Prior inference-time stitching re-noises and splits a plan before re-denoising its halves under signed-distance and smoothness guidance.DetAug removes the split and uses geometry to rank candidates rather than construct trajectories.Source: C. Trajectory Stitching and Splitting · p. 3

A. Overview

  • DetAug augments free-space demonstrations with random arcs, records their parameters as labels, and trains a flow-matching policy conditioned on those labels.The label is injected via AdaLN-Zero.Source: A. Overview · p. 3
  • At inference, DetAug samples K plans with random labels from the training support, scores them with an analytic collision cost, and executes the best trajectory.The method does not require training-time obstacle geometry or assume coverage by the original prior.Source: A. Overview · p. 3

B. Architecture

The policy uses a DiT-style transformer with conditional flow matching, and injects flow time and the augmentation label through AdaLN-Zero. Detour labels describe arcs substituted for transit segments while interaction windows remain unchanged.

B. Architecture

  • A DiT-style transformer is trained with conditional flow matching, with flow time and the augmentation label injected through AdaLN-Zero.The same conditioning choice makes it straightforward to graft the augmentation label onto a pretrained policy.Source: B. Architecture · p. 3

C. Detour Augmentation

  • Each grasp has approach and carry transit phases; sampled circular arcs replace their interior segments while grasp and release windows remain fixed.Arc half-angle φ and plane rotation θ define the detours, and infeasible or nonsmooth samples are rejected.Source: C. Detour Augmentation · p. 3, Figure 3 · p. 3
  • An arc’s label is φ(cos θ, sin θ), yielding a four-dimensional descriptor per grasp by concatenating the approach and carry labels.The descriptor is zero for unbent demonstrations; multi-grasp tasks concatenate labels for separately augmented segments.Source: C. Detour Augmentation · p. 3

D. Inference-Time Selection

At inference, DetAug samples augmentation labels, generates candidate trajectories, and selects one using collision cost. It refreshes the candidates at each replan and scores penetration rather than clearance.

D. Inference-Time Selection

  • Each replan samples K labels uniformly from the training support, generates trajectories conditioned on the observation and optional goal, and selects using collision cost.The batch is refreshed at every replan rather than latching the first choice; latching roughly halved collision-free success.Source: D. Inference-Time Selection · p. 3
  • The collision cost checks sampled points on the robot’s surface against an obstacle signed distance field and scores penetration rather than clearance.Clearance-based scoring tended to encourage overly circuitous paths and often hurt task success.Source: D. Inference-Time Selection · pp. 3–4
  • Table I reports SafeLIBERO benchmark results aggregated by suite, with Clean, Total, and Avoid denoting collision-free task success, task success irrespective of collisions, and collision avoidance.Its entries are percentages averaged over Level I and Level II scenarios.Source: D. Inference-Time Selection · p. 4, Table I · p. 4

IV. EXPERIMENTAL RESULTS

The SafeLIBERO experiments test obstacle avoidance on simulated tasks whose training scenes come from standard LIBERO and whose test scenes receive random obstacles. DetAug leads the reported comparison in collision-free success, while inference-time alternatives often avoid obstacles without completing the task.

A. SafeLIBERO Benchmark

  • SafeLIBERO uses standard LIBERO scenes for training and inserts random obstacles at test time, with difficulty varying across scenarios.DemoGen represents train-time obstacle access; AEGIS and CAPE represent inference-time approaches.Source: A. SafeLIBERO Benchmark · p. 4
  • The experiments use state-based policies for quick iteration, and omit about 20% of tasks because no dataset interaction segments were collision-free for those obstacles.The omitted tasks yielded close to 0% success for all methods.Source: A. SafeLIBERO Benchmark · p. 4
  • 52.2% clean success rate gave DetAug an advantage of more than 20pp over the next best method on SafeLIBERO.AEGIS and CAPE had high avoidance but low success and often timed out; DemoGen had low avoidance despite access to obstacle geometry.Source: A. SafeLIBERO Benchmark · p. 4

B. Adapting a Pre-Trained Policy

In a single Newton pick-and-place task with a test-time cylindrical obstacle, the adapter-only comparison favors DetAug, while full fine-tuning changes the outcome. The comparison tests adding obstacle avoidance to an existing policy using AdaLN conditioning.

B. Adapting a Pre-Trained Policy

  • The comparison uses a pre-existing policy simulated with the state-based architecture and a single Newton pick-and-place task with a cylindrical test-time obstacle.AdaLN injects the augmentation label for DetAug and a point-cloud embedding for DemoGen.Source: B. Adapting a Pre-Trained Policy · p. 4
  • Adapter-only training updates the label encoder and AdaLN modulation layers while freezing the backbone; full fine-tuning also trains the main policy.The adapter-only setting trains about 3M parameters.Source: B. Adapting a Pre-Trained Policy · pp. 4–5
  • 35% final success for DetAug versus DemoGen’s 2% and the base policy’s 6% in the adapter-only setting.With full fine-tuning, DemoGen beat DetAug in success and matched its collision rate, but required more training steps.Source: B. Adapting a Pre-Trained Policy · p. 5, Figure 4 · p. 5
  • Full fine-tuning lets point-cloud conditioning match or outperform DetAug, but training billion-parameter VLAs can be prohibitively expensive and take many days to converge.The paper therefore distinguishes this outcome from the limited-compute adapter-only setting.Source: B. Adapting a Pre-Trained Policy · p. 5

C. Selection vs Guidance

On the SafeLIBERO spatial suite, DetAug’s label selection outperforms guidance using the same collision method. The authors hypothesize that a lower-dimensional search and conditioning on clean-sample parameters contribute to the difference.

C. Selection vs Guidance

  • 26pp lower collision-free success for null-token guidance than DetAug’s label selection on the SafeLIBERO spatial suite.Both approaches use the same collision method; guidance is applied without selection, and the mixed variants also perform worse.Source: C. Selection vs Guidance · p. 5
  • The authors hypothesize that searching a 4-dimensional label space is easier than searching the full 52-dimensional state and action space.They also propose that labels represent clean-sample parameters, whereas guidance acts on intermediate noisy samples that are off-manifold.Source: C. Selection vs Guidance · p. 5

D. Effect of the Number of Sampled Labels

On real hardware, DetAug outperforms the compared methods, while inference-time methods struggle on large detours. In the SafeLIBERO spatial suite, random labels outperform zero-valued labels, and increasing the number of sampled labels improves success at higher inference cost.

D. Effect of the Number of Sampled Labels

E. Hardware Experiments

V. CONCLUSIONS

DetAug adapts manipulation policies for zero-shot obstacle avoidance without needing test-time environment knowledge during training. The paper reports strong simulation results and hardware performance against methods that use obstacles during training or steer policies at inference.

  • DetAug adapts manipulation policies for zero-shot obstacle avoidance without requiring knowledge of the test-time environment during training.Source: §V · p. 6
  • On a broad suite of simulation tasks, DetAug significantly outperforms prior methods, and selection over augmentation labels outperforms guidance on the same policy.Source: §V · p. 6
  • DetAug matches or exceeds an obstacle-conditioned baseline on real hardware, where inference-time methods collapse on large detours.Source: §V · p. 6
  • The paper identifies scaling to a large VLA model and tasks requiring greater dexterity as future work.Source: §V · p. 6

Related papers

Loading 2609.18395v1…