Source-linked AI summary
Learning Navigation from Multimodal Demonstrations with Geometric Candidate Modes
Chengyang Peng, Zhihao Zhang, Upama Pant, Yiming Wang, Keith A. Redmill, Ayonga Hereid
TL;DR
Offline navigation demonstrations can contain multiple valid routes from similar observations, while action regression can average alternatives and current actions may overlap across routes. This paper computes candidate modes from local geometry and trains a hierarchical policy to select and execute them. In simulation, it outperforms four offline learning baselines and a heuristic using the same modes, and the simulation-trained policy also runs on a bipedal robot.
Problem
Navigation demonstrations are multimodal, and similar states can lead to different routes whose current actions overlap, making the alternatives difficult to infer from actions alone.
Source: §I · p. 1Method
The policy computes candidate modes from local geometry and learns from demonstrations which mode to select and how to execute it.
Source: §I · pp. 1–2, A. Candidate Modes · pp. 2–3Results
96.5% of seen and 95.2% of unseen cases succeed, versus at most 43% and 20% for offline baselines and 79.2% and 66.7% for a heuristic using the same modes.
Source: §I · p. 2Takeaways & Limitations
The simulation-trained policy ran on a LimX Tron 1 bipedal robot without retraining.
Source: §V · p. 8
Abstract
from arXiv · showhide
Offline navigation demonstrations are multimodal, because one observation often allows several valid actions, such as passing an obstacle on either side. Behavior cloning averages these actions, potentially leading to unsafe routes. Multimodal policies must discover the alternatives from the data, but similar states can lead to different routes whose current actions overlap. In navigation, the alternatives have a geometric cause and can be computed from one observation. We propose a hierarchical neural-network policy that computes candidate modes from local geometry and learns from demonstrations which mode to follow and how. A high-level policy selects a mode using the goal and its previous choice among a set of candidate modes, and a low-level policy turns it into an action. In simulation, the policy succeeds in over 95\% of seen and unseen test cases, compared with at most 43\% and 20\% for four offline learning baselines and 79\% and 67\% for a heuristic that uses the same modes without learning. The simulation-trained policy also transfers to a LimX Tron 1 bipedal robot.
I. INTRODUCTION
Navigation demonstrations contain several valid routes, but single-action regression can average them into unsafe actions, while different routes may have overlapping current actions. The paper uses geometry to enumerate alternatives and learns a hierarchical policy to select and execute them, reporting strong simulation results and transfer to a bipedal robot.
I. INTRODUCTION
- Single-action regression averages valid alternatives, potentially directing a robot between left- and right-side obstacle detours into the obstacle.Different routes can also have overlapping current actions, so actions alone do not always reveal the route alternatives.Source: §I · p. 1
- Nearby obstacles divide free space into passages that geometry can enumerate from one observation, but geometry alone lacks route knowledge beyond the observed region and memory of prior choices.The demonstrations come from a planner that sees the whole map and follows a route from start to goal.Source: §I · p. 1
- The proposed hierarchy computes candidate modes from local geometry, then learns from demonstrations which mode to select and how to execute it.The high-level policy conditions on the goal and previous mode; the low-level policy produces a continuous action.Source: §I · pp. 1–2, Figure 2 · p. 2
- 96.5% of seen and 95.2% of unseen cases succeed, versus at most 43% and 20% for offline baselines and 79.2% and 66.7% for a same-mode heuristic.The simulation-trained policy also ran on a LimX Tron 1 without retraining or environment-specific tuning.Source: §I · p. 2
- The evaluation separates geometry's contribution from learning's: the same-mode heuristic trails the learned policy, which also makes fewer mode switches and abrupt action changes.The paper additionally reports hardware experiments on a LimX Tron 1 bipedal robot.Source: §I · p. 2
II. GEOMETRIC REPRESENTATION OF NAVIGATION ALTERNATIVES
The method represents goal-conditioned navigation with local observations and constructs a variable-size set of candidate modes from obstacle geometry, without learning those alternatives from demonstrations.
II. GEOMETRIC REPRESENTATION OF NAVIGATION ALTERNATIVES
- Navigation uses a local occupancy map and proprioception with a robot-frame goal; candidate modes are computed from the observation and goal, while demonstrations train mode choice and execution.The action is a desired forward displacement and heading direction, converted to a velocity command by an MPC planner.Source: §II · p. 2
A. Candidate Modes
- Tangent lines at closest obstacle points and the observation-window edges define safe intersections; retained points become modes, with a goal mode added when its direction is unblocked.The local observation window is a 3 m × 3 m square, and the number of candidate modes varies with obstacle layout.Source: A. Candidate Modes · p. 2
- The four mode types represent passage between obstacles, passing one obstacle on either side, or moving toward the goal when that direction is unblocked.Type-0 modes also carry a binary indicator of whether the point lies inside the passage.Source: A. Candidate Modes · p. 3, Figure 3 · p. 3
B. Mode Encoding and Reference Action
Each candidate mode is encoded with geometric and obstacle features, and also supplies a reference action that points toward its candidate point.
B. Mode Encoding and Reference Action
- Each mode is represented by 22 numbers that describe its point, adjacent obstacles, passage, initialization status, and goal features.The obstacle features include the mode type, closest-point geometry for obstacles on either side, masks, and a passage indicator.Source: B. Mode Encoding and Reference Action · p. 3
- The examples show that similar states can lead to distinct future routes even when current actions overlap and vary within each route group.Fig. 4 illustrates an earlier case where current actions form separate groups; Fig. 5 shows the intermediate case with overlapping actions.Source: Figure 5 · p. 3, Figure 4 · p. 3
- A mode's reference action moves toward its point, using the point's direction as heading and its distance, capped at 1.5 m, as displacement.The reference action is computed from geometry alone.Source: B. Mode Encoding and Reference Action · p. 3
III. HIERARCHICAL POLICY LEARNING
The hierarchical policy separates selecting a candidate mode from choosing the action associated with it, addressing gaps in a simple goal-nearest-mode rule.
III. HIERARCHICAL POLICY LEARNING
- The policy learns from demonstrations by having a high-level component select a mode and a low-level component turn that selection into an action.The planner that generated the demonstrations had access to the whole map and followed a route to the goal.Source: §III · p. 3
- Choosing the mode whose point is closest to the goal cannot identify which passage leads beyond the observation window or remember the previous choice.The simple rule also provides only a coarse direction toward the mode point, so its actions can change abruptly.Source: §III · p. 3
- The demonstrations supply route information missing from local geometry, motivating learned mode selection and execution.Source: §III · p. 3
A. Demonstrations and Mode Labels
The authors use full-map planner demonstrations and future motion to assign geometric mode labels without manual annotation. The labels account for displacement, heading, and whether motion stays within a mode’s region.
A. Demonstrations and Mode Labels
- About 10,000 planner-generated trajectories provide demonstrations in a 5 m × 5 m workspace with varied obstacle shapes.The RRT-based planner sees the whole map; each trajectory contains approximately 130 walking steps.Source: A. Demonstrations and Mode Labels · p. 3
- Routes can diverge even when their current actions overlap, so the authors define mode labels from future motion and obstacle geometry.Demonstrations show distinct actions near trajectory starts but overlapping, widely varying actions later.Source: A. Demonstrations and Mode Labels · pp. 3–4
- Each state is matched to a mode using the next 8 steps’ displacement and heading change, plus whether that motion remains within the mode’s region.The region cost uses an angular sector bounded by adjacent-obstacle rays for Types 0–2 and a ±30° cone for Type 3; weights are set by hand to (0.3, 0.3, 0.4).Source: A. Demonstrations and Mode Labels · p. 4
- The lowest-cost mode becomes the label, and labels require no manual annotation and are used only during training.Source: A. Demonstrations and Mode Labels · p. 4
B. Policy Architecture and Losses
The policy separates mode selection from action generation: the high-level policy chooses among geometric modes, and the low-level policy produces an action for the selected mode. Training adds a mode-reference action target whose weight depends on how reliable that reference is.
B. Policy Architecture and Losses
- The policy first selects a mode and then predicts an action for that mode, avoiding the conditional mean that averages actions across modes.Source: B. Policy Architecture and Losses · p. 4
- The high-level policy scores modes using the observation, goal, and previous mode, whose inclusion gives it memory to help maintain its choice across steps.At runtime, the most likely valid mode is selected; the variable-size mode list is padded to 12 with a mask, and the tested mode count does not exceed that limit.Source: B. Policy Architecture and Losses · p. 4
- The low-level policy receives the current observation and one mode, predicts a continuous action, and does not receive the goal directly.During training it receives the demonstration-matched mode, so it does not experience high-level selection errors.Source: B. Policy Architecture and Losses · p. 4
- An auxiliary loss trains the low-level policy toward a mode reference action, with its weight reflecting the reference’s reliability.For Types 0–2, the reference is more reliable near obstacles and its weight rises from 0 at 0.8 m to 1 at 0.3 m; Type 3 uses a goal-directed reference with weight 1.Source: B. Policy Architecture and Losses · p. 4
- Because the auxiliary weight is at most 1, the demonstrated action retains at least half the weight in the combined target.Even for Type 3, the target is the midpoint of the demonstrated action and reference action, not the reference alone.Source: B. Policy Architecture and Losses · p. 5
IV. SIMULATION AND HARDWARE EXPERIMENTS
The experiments assess the policy in simulation and on a bipedal robot. The simulation comparisons separate the contribution of learned mode selection from that of the geometric modes, while hardware tests examine simulation-to-robot transfer.
IV. SIMULATION AND HARDWARE EXPERIMENTS
- The simulation evaluation compares the policy with offline learning methods that do not use geometric modes.Source: §IV · p. 5
- A separate simulation comparison uses model variants, including a heuristic that applies the same modes without learning.Source: §IV · p. 5
- Hardware experiments test whether a policy trained only in simulation runs on a real robot with onboard perception.Source: §IV · p. 5
A. Experimental Setup
The experiments use five-seed training, held-out trajectories, and seen and unseen maps with different obstacle counts. Closed-loop trials measure success and failure outcomes alongside path, step, smoothness, and computation measures.
A. Experimental Setup
- The policies are trained separately with a trajectory-level 15% validation split, and results report the mean and standard deviation across five random seeds.Training takes approximately 1.2 hours per complete hierarchical policy on a single NVIDIA T1200 Laptop GPU.Source: A. Experimental Setup · p. 5
- Seen tests comprise 250 cases, while 30 unseen maps use 6, 10, or 14 obstacles, with gaps wider than the narrowest seen-map gaps.Seen cases include 50 obstacle-free maps and 100 each with four and eight obstacles.Source: A. Experimental Setup · p. 5
- All methods run closed loop on the same starts and goals, receiving new observations generated by their own prior actions.A trial ends at the goal, on collision, or after 200 steps.Source: A. Experimental Setup · p. 5
- The evaluation reports success, collision, and timeout rates, plus path and step ratios, action jumps, mode switches, large turns, and computation time.Path and step ratios compare cases completed by both the method and Ours; jumps and switches are counts per 100 steps.Source: A. Experimental Setup · p. 5
B. Comparison with Offline Learning Baselines
Across shared demonstrations and inputs, Ours outperforms four offline learning baselines and runs in real time. Comparisons with variants using the same modes show that mode selection and the auxiliary loss contribute to robust navigation, while learning from demonstrated actions improves motion efficiency.
B. Comparison with Offline Learning Baselines
- 96.5% seen and 95.2% unseen success for Ours, versus no offline baseline above 43% and 20%, respectively.Almost all baseline failures are collisions; on unseen maps, baseline collision rates are 80% or higher versus 2.6% for Ours.Source: B. Comparison with Offline Learning… · p. 5, Table I · p. 6
- Among jointly completed cases, baseline path ratios are 0.88–1.12 and step ratios are mostly 1.01–1.18; Ours runs in 2.9 ms on seen and 3.4 ms on unseen maps.The ratios cover only cases that a baseline also completes, at most 43% of seen and 20% of unseen cases; DP takes about 100 ms per step.Source: B. Comparison with Offline Learning… · p. 6
C. Comparison with Model Variants
- 25.8% seen and 10.7% unseen success for w/o Hier. shows that giving candidate modes to one network without mode selection is insufficient.Methods that select a mode, including the Heuristic, succeed more often (62.0–98.0%).Source: C. Comparison with Model Variants · p. 6, Table II · p. 7
- Without the auxiliary loss, success falls to 70.4% seen and 62.0% unseen, mostly because of collisions.The variant sets λaux to zero and trains the low-level policy only from demonstrated actions; the paper supports using the reference action as a second target near obstacles.Source: C. Comparison with Model Variants · p. 6
- Mode-act.-only reaches 97.8% seen and 98.0% unseen success with almost no collisions, but its paths are 9% and 7% longer and it makes more action jumps.This variant trains the low-level policy only toward reference actions; adding demonstrated actions yields shorter, smoother motion with 3.2% and 2.6% collisions while success stays above 95%.Source: C. Comparison with Model Variants · p. 6
- 79.2% seen and 66.7% unseen success for the Heuristic, below Ours at 96.5% and 95.2%, despite using the same modes.The Heuristic also makes more mode switches and action jumps; Table II compares it with variants using the same modes and demonstrations.Source: C. Comparison with Model Variants · p. 6, Table II · p. 7
- In the seen example, the Heuristic collides after 108 steps, while Ours reaches the goal in 103; in the unseen example, both succeed but the Heuristic takes 143 steps versus 98.The unseen paths are similar in length, 5.9 and 5.7 m, but the Heuristic makes 40 action jumps versus 1 for Ours.Source: C. Comparison with Model Variants · pp. 6–7, Figure 6 · p. 7, Table III · p. 7
D. Hardware Experiments
The simulation-trained policy ran on a LimX Tron 1 bipedal robot without retraining or environment-specific tuning. It navigated indoor settings with different obstacles and levels of difficulty.
D. Hardware Experiments
- The simulation-trained policy transferred to real indoor environments without retraining or tuning for each environment.The robot built a local occupancy map from onboard LiDAR and ran the policy in closed loop at 24 Hz.Source: D. Hardware Experiments · pp. 7–8
- The robot consistently reached the laboratory goal, passing obstacles and changing direction as new obstacles entered its observation window.The real maps included furniture, boxes, and walls, as well as perception noise.Source: D. Hardware Experiments · p. 7
- Figure 8 shows deployments in a public study area, a table-avoidance-and-return task, and a doorway leading into a corridor.The corridor is a narrow passage followed by long walls.Source: D. Hardware Experiments · p. 7, Figure 8 · p. 8
V. CONCLUSION
The paper concludes that local geometry can supply candidate navigation modes while learning selects and executes them. The approach outperformed offline-learning baselines and a heuristic in simulation, and the simulation-trained policy ran on a bipedal robot without retraining.
V. CONCLUSION
- The policy succeeded more often in simulation than offline-learning baselines and a heuristic using the same modes, while colliding less and switching modes less often than the heuristic.The authors identify extending the approach to moving obstacles as future work.Source: §V · p. 8
- Local geometry supplies candidate modes, and learning selects and executes them for offline navigation.Source: §V · p. 8
- The policy trained in simulation also ran on a bipedal robot without retraining.Source: §V · p. 8