PaperlayerView exact revision on arXiv

Source-linked AI summary

From Failures to Learnable Worlds: Policy-Conditioned Environment Generation for Navigation Agents

Yue Zhang, Han Lin, Zun Wang, Yidong Huang, Zaid Khan, Mohit Bansal

arXiv:2610.19420v1cs.ROcs.AIcs.CV

TL;DR

Existing environment-generation pipelines often target generic environment properties, although different pretrained policies can struggle for different reasons. WEDGE diagnoses validated policy weaknesses and constructs learnable training environments around them; weakness-conditioned adaptation outperforms matched alternatives on ProcTHOR, with gains for Uni-NaVid and Gemini also extending to HM3D.

  • Problem

    Environment-generation pipelines largely target realism, diversity, or task coverage, while giving limited consideration to what a particular policy has mastered or where it still struggles.

    Source: §1 · pp. 1–2

  • Method

    WEDGE validates weakness features derived from policy rollouts, constructs environments in the diagnosed weak regime, and uses five-rollout verification to select candidates with two or three successes.

    Source: §3.2 · p. 5, §3.3 · p. 6

  • Results

    Across three navigation agents, weakness-conditioned adaptation outperforms matched random and hard-example alternatives on ProcTHOR, and Uni-NaVid and Gemini gains also extend to HM3D after ProcTHOR adaptation.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Policy failures can guide construction of training environments around what each policy still needs to learn, and re-diagnosis can guide continued adaptation across rounds.

    Source: §5 · p. 10

    Evaluation is limited to ObjectNav in indoor 3D environments, using ProcTHOR for diagnosis and generation and HM3D for cross-domain evaluation.

    Source: §A.4 · p. 18

Abstract

from arXiv · show

Simulated environments enable scalable training data generation for embodied agents, but more data does not necessarily provide what a particular agent needs to learn. Existing environment-generation pipelines largely optimize generic properties such as realism, diversity, or task coverage, while different pretrained policies can fail for different environmental reasons. We therefore introduce WEDGE, a framework for weakness-guided navigation learning in which the failure modes of a pretrained agent guide the generation of training environments at each adaptation round. Given a pretrained policy, WEDGE uses an LLM to propose executable weakness features from contrasting successful and failed rollouts, automatically validates them on the diagnostic set, and selects the strongest validated weakness based on its success-rate gap. The validated weaknesses are then mapped to the environmental factors that control them. WEDGE varies these factors to construct environments in the difficulty regimes where the policy struggles, producing targeted training experience for adaptation. Across three navigation agents with different adaptation mechanisms, we find that policies exhibit distinct weakness profiles and benefit from different training environments. Weakness-conditioned adaptation consistently outperforms matched random and hard-example alternatives on ProcTHOR, while the resulting gains for Uni-NaVid and Gemini also extend to HM3D after ProcTHOR adaptation. Iterative re-diagnosis further shows that dominant weaknesses can persist or shift as the policy improves, providing updated guidance for subsequent environment construction. Together, these results show that policy failures are not only diagnostic signals, but actionable guidance for constructing training environments that target a policy's current weaknesses.

1 INTRODUCTION

Existing environment generators often target realism, diversity, or task requirements rather than what a particular policy has not mastered. WEDGE diagnoses policy-specific failures and uses them to guide training-environment construction and adaptation.

1 INTRODUCTION

  • Existing generation pipelines increasingly support realism, diversity, and task-conditioned construction, but give limited consideration to what a current policy has mastered or where it still struggles.The motivating question is how to generate environments targeting what a particular agent has not yet mastered.Source: §1 · pp. 1–2
  • Policies solving the same task can struggle for different environmental reasons, making a shared training environment redundant for one agent but informative or too difficult for another.Examples include failures in multi-room navigation versus failures when targets are poorly visible.Source: §1 · p. 2
  • WEDGE contrasts successful and failed episodes, uses an LLM to propose executable weakness features, validates them, and maps retained weaknesses to controllable environmental factors and failure-associated regimes.The policy is rolled out in candidate environments to retain regimes that are challenging but solvable, then adapted with its original learning mechanism.Source: §1 · pp. 2–3
  • Across three agents, weakness-conditioned adaptation outperforms matched random and hard-example alternatives on ProcTHOR, with Uni-NaVid and Gemini gains also extending to HM3D.The agents’ dominant weaknesses can persist or shift after adaptation, and re-diagnosis guides subsequent environment generation.Source: §1 · p. 3

2 RELATED WORK

Prior work has expanded the scalability and controllability of embodied-AI environments and explored adaptive, learner-informed generation. WEDGE focuses instead on diagnosing policy-specific failure conditions within a task and using them to construct targeted, learnable environments.

2 RELATED WORK

  • Earlier environment-design work expands scene construction, simulation-ready generation, training experience, and robustness, while WEDGE focuses on what to construct for a particular pretrained policy.The distinction is policy-specific weakness diagnosis rather than environment scalability or controllability alone.Source: §2 · pp. 3–4
  • Adaptive curricula and environment-design methods use learner feedback, LLM configuration, performance-adaptive curricula, or trajectory-informed edits, while WEDGE diagnoses measurable policy-specific weaknesses within the same navigation task.WEDGE validates those weaknesses before using them to choose environmental factors and target regimes.Source: §2 · pp. 3–4

3 METHOD

WEDGE closes the loop from a pretrained ObjectNav policy’s behavior to environment construction and policy adaptation. It discovers and validates policy-specific weaknesses, constructs environments in the associated regimes, filters them for learnable difficulty, and adapts policies using their original learning mechanisms.

3.1 PROBLEM SETUP

  • ObjectNav episodes combine an environment, initial state, and target category; executing the pretrained policy produces a rollout and binary task outcome.The simulator supplies controllable layouts, object arrangements, and other task-relevant properties.Source: §3.1 · p. 4

3.2 LLM-GUIDED DISCOVERY OF EXECUTABLE WEAKNESSES

  • WEDGE samples 15 successful and 15 failed episodes, represents them with shared measurable primitives, and prompts an LLM to propose executable weakness features.Features are checked for coverage, non-redundancy, predictiveness, and incremental value; validated weaknesses distinguish low- from high-feature-value episodes by a sufficient success-rate gap.Source: §3.2 · p. 5

3.3 WEAKNESS-CONDITIONED ENVIRONMENT GENERATION

  • For each adaptation round, WEDGE selects the weakness with the largest success-rate gap and modifies the episode factors on which that feature depends.It retains candidates in the diagnostic failure-associated range and generates additional layouts when needed to find enough qualifying environments.Source: §3.3 · pp. 5–6
  • Five policy rollouts filter weakness-matched candidates to those with two or three successes, excluding episodes that are almost always solved or almost never solved.The verification rollouts select episodes; they are not used as training trajectories.Source: §3.3 · p. 6

3.4 TRAINING DATA CONSTRUCTION AND POLICY ADAPTATION

  • WEDGE changes which episodes each policy learns from while preserving its original adaptation mechanism.It continues reinforcement learning with the original objective, fine-tunes imitation policies on successful self-rollouts, or uses successful trajectories as in-context examples for models without parameter access.Source: §3.4 · pp. 6–7

4 EXPERIMENT

The experiments test WEDGE across three navigation agents, finding distinct policy weakness profiles and better adaptation from weakness-conditioned experience than matched alternatives. Further analyses examine cross-simulator performance, iterative re-diagnosis, gains in targeted regimes, calibration, and search efficiency.

4.1 EXPERIMENTAL SETUP

  • The evaluation covers DD-PPO, Uni-NaVid, and Gemini using ProcTHOR for diagnosis and environment construction, with matched-budget random and hard-mined baselines.The agents use different training or adaptation setups, and Uni-NaVid and Gemini are also evaluated on HM3D.Source: §4.1 · pp. 7–8

4.2 POLICY-SPECIFIC WEAKNESS DISCOVERY

  • The agents have distinct diagnosed weaknesses: DD-PPO’s is target visibility, while blind path ratio leads for Uni-NaVid and Gemini.The reported SR gaps are 28.5 for DD-PPO, 37.3 for Uni-NaVid, and 37.6 for Gemini.Source: §4.2 · p. 8

4.3 WEAKNESS-CONDITIONED ADAPTATION

  • Weakness-conditioned adaptation outperforms matched random and hard-example alternatives for all three agents on ProcTHOR, with Uni-NaVid and Gemini also improving on HM3D.For Uni-NaVid, HM3D SR reaches 73.4 after ProcTHOR adaptation; Gemini reaches 63.5 SR on HM3D.Source: §4.3 · p. 8
  • DD-PPO and Uni-NaVid retain their dominant weakness across adaptation rounds, while Gemini’s shifts from blind path ratio to target room remoteness.Uni-NaVid continues improving from 54.5 to 55.7 ProcTHOR SR; DD-PPO’s SR and SPL effectively saturate after its first round.Source: §4.3 · p. 8, Figure 4 · p. 9

4.4 ANALYSIS AND ABLATIONS

  • Adaptation gains are larger in diagnosed weakness regimes, including Gemini’s +7.0 SR on long-blind-path episodes and DD-PPO’s small, hard-to-see targets.Uni-NaVid and Gemini gains are compared across blind-path regimes; DD-PPO gains are reported by target category.Source: §4.4 · p. 9
  • Finding the same number of hard target placements requires roughly half as many candidates and policy rollouts with weakness guidance as with unguided hard-example mining.The comparison is 4.2k versus 8.4k candidates, or 21k versus 42k policy rollouts.Source: §4.4 · p. 9
  • Rollout calibration focuses training on intermediate-difficulty environments: DD-PPO’s 16% calibrated subset reaches 85.4 SR versus 84.4 using the full set.For Uni-NaVid, increasing the share of hardest episodes does not improve performance and slightly degrades results.Source: Table 2 · p. 9, §4.4 · p. 9

5 CONCLUSION

The conclusion presents WEDGE as a framework that uses policy-specific failures to guide navigation training-environment generation. It reports adaptation gains across agents, transfer across simulators, and re-diagnosis as support for using failures to guide continued adaptation.

5 CONCLUSION

  • WEDGE diagnoses policy-specific failure conditions and uses them to guide navigation training-environment construction.The framework closes the loop between diagnosis, environment construction, and policy adaptation.Source: §5 · p. 10
  • Across different agents and learning mechanisms, WEDGE improves adaptation and its gains can transfer across simulators.Source: §5 · p. 10
  • Re-diagnosis can support continued adaptation over multiple rounds by updating guidance as policy weaknesses persist or shift.Source: §5 · p. 10

A.1 WEAKNESS FEATURE VALIDATION

Candidate weakness features are evaluated through sequential checks for coverage, distinctiveness, predictive success-rate separation, and incremental value. A feature is retained only if it passes every criterion.

A.1 WEAKNESS FEATURE VALIDATION

  • Validation rejects a candidate as soon as it fails any of four sequential criteria: coverage, non-redundancy, predictiveness, or incremental value.Candidates are evaluated over the diagnostic dataset.Source: §A.1 · p. 14
  • A feature must be computable for nearly all diagnostic episodes and must not be constant on the episodes where it is defined.The coverage threshold is 0.95.Source: §A.1 · p. 14
  • A candidate must avoid strong correlation with previously retained features, with non-redundancy checked using Spearman correlation on the diagnostic set.The correlation threshold is 0.85, and this check is skipped when no features have yet been retained.Source: §A.1 · p. 14
  • A feature must separate bottom- and top-tercile episodes by a significant success-rate gap and preserve the gap's direction on held-out data.The required gaps are at least 10 percentage points in the diagnostic split and 5 points on the held-out split.Source: §A.1 · p. 14
  • A new feature must improve five-fold cross-validated AUC for a logistic-regression predictor of episode success using retained features.The incremental AUC threshold is 0.01, with baseline AUC set to 0.5 when no features are retained.Source: §A.1 · p. 14

A.2.1 COMPUTATIONAL COST.

Weakness discovery has modest overhead relative to environment evaluation and adaptation, while weakness-guided filtering reduces the candidate search needed for hard target placements. Adaptation data and compute requirements differ across the three agents.

A.2.1 COMPUTATIONAL COST.

  • Weakness-guided filtering needed 4.2k candidates and 21k rollouts to obtain the same number of hard target placements, versus 8.4k candidates and 42k rollouts for unguided mining.Discovery uses logged diagnostic episodes, runs on CPU, and requires about ten LLM calls per agent; rollout verification and adaptation dominate cost.Source: §A.2.1 · p. 15
  • A complete adaptation round requires about one GPU-day for DD-PPO, two GPU-days for Uni-NaVid, and half a day plus about $110 in API cost for Gemini.The dominant costs are rollout-based candidate verification and subsequent adaptation.Source: §A.2.1 · p. 15
  • The adaptation pools differed by agent: DD-PPO used 4,215 placements, Uni-NaVid assembled 51,162 training samples, and Gemini used 400 practice episodes.Uni-NaVid's pool included 1,630 successful mined trajectories and 850 successful generated trajectories; Gemini's practice set yielded 64 successful trajectories, one selected as its worked example.Source: §A.2.1 · p. 15

A.2.2 GEMINI NAVIGATION AGENT

Gemini navigates ProcTHOR using four directional views, recent action history, and constrained movement and stopping rules. Its in-context adaptation adds successful weakness-matched trajectories as worked examples, including examples collected after re-diagnosis.

A.2.2 GEMINI NAVIGATION AGENT

  • Gemini receives four 90°-separated RGB views and its previous 16 decisions, then predicts target visibility and an action from five choices.Directional actions orient the agent and execute up to three 0.25 m steps; episodes end after at most 60 decisions.Source: §A.2.2 · p. 15
  • Gemini may choose END only when it predicts the target visible, and a second model call checks that the target is identifiable and approximately within 1 m.A rejected stop consumes one decision and enters the action history.Source: §A.2.2 · p. 15
  • For weakness-conditioned adaptation, one successful trajectory from 400 weakness-matched practice episodes is compressed into a worked example and appended to the navigation prompt.The example includes a two-sentence explanation generated by Gemini-2.5-Pro.Source: §A.2.2 · p. 15
  • Iterative adaptation re-diagnoses Gemini after the first worked example, gathers a new weakness-matched practice set, and appends a second successful trajectory.The Round 2 prompt contains both round-specific examples.Source: §A.2.2 · pp. 15–16
  • The prompt requires a two-line visibility-and-action response, exploration toward less explored space, and changing direction after blocked paths or loops.It permits END only when the target is visible and asks the agent to move closer when the visible target is too far away.Source: §A.2.2 · p. 16
  • Worked examples illustrate successful behavior through sustained movement across rooms, recovery from blocked paths, and approaching closer after a rejected stop.The examples describe continuing through unexplored space rather than repeatedly searching nearby areas.Source: §A.2.2 · p. 16

A.3 PROMPT FOR WEAKNESS FEATURE PROPOSAL

The feature-proposal prompt asks an LLM to derive executable numeric weakness features from sampled successful and failed episodes and available simulator measurements. The appendix also states the framework's evaluation scope and implementation assumptions.

A.3 PROMPT FOR WEAKNESS FEATURE PROPOSAL

  • The prompt asks for new numeric environmental features that may predict ObjectNav failure and guide training-environment generation.Proposals use contrasting successful and failed episodes.Source: §A.3 · p. 17
  • The supplied episode fields cover scene, target, and planner properties, including room counts, target visibility, room hops, and path measurements.The prompt provides 15 sampled successful and 15 sampled failed episodes for contrast.Source: §A.3 · p. 17
  • The LLM must return exactly five Python functions, each mapping one episode dictionary to a numeric feature using only listed fields and guarding against missing values.Features should preferably reflect controllable geometry, layout, object placement, or combinations of those properties.Source: §A.3 · p. 17
  • Scene, target, and path primitives depend on layout and the ground-truth shortest path, making them controllable during generation.The measurement vocabulary summarizes every evaluated episode.Source: Table 5 · p. 18
  • The prompt notes that occlusion degree is logged as both a mean and a maximum over standing spots.Source: §A.3 · p. 18

A.4 LIMITATIONS

  • Evaluation is limited to indoor 3D ObjectNav, using ProcTHOR for diagnosis and generation and HM3D for cross-domain evaluation.The authors identify additional navigation tasks, simulators, and real-world settings as future extensions.Source: §A.4 · p. 18
  • The implementation assumes measurable simulator primitives and controllable environment factors, and targets only the top-ranked weakness per adaptation round.Coordinating multiple complementary weaknesses within a round is left for future work.Source: §A.4 · p. 19

Related papers

Loading 2610.19420v1…