PaperlayerView exact revision on arXiv

Source-linked AI summary

DAWN: Noise-Robust Quadruped Parkour via Depth-Denoising World Models

Yohan Choi, Min-Jun Kim, Jin-Sung Kim, Yong-Jae Kim, Youn-Hee Han

arXiv:2609.29092v1cs.ROcs.AIcs.CVcs.LG

TL;DR

Vision-based locomotion commonly trains on clean depth and relies on deployment filters, while depth-noise robustness has received limited world-model treatment. DAWN trains an RSSM with noisy-input/clean-target denoising and contrastive latent alignment. It reports zero-shot quadruped parkour from raw depth without manual filter calibration, with denoising and alignment contributing complementary gains.

  • Problem

    Vision-based locomotion methods commonly train with clean depth and rely on deployment filters, while integrating depth-noise robustness into world models remains unexplored.

    Source: §I · p. 1, B. Depth Sim-to-Real Transfer · p. 2

  • Method

    DAWN modifies a world model with noisy-depth input and clean-depth reconstruction, and contrastively aligns noisy and clean latent representations.

    Source: §I · p. 1, §V · p. 7

  • Results

    96.9% average simulation success across stairs, gaps, and steps; zero-shot Go1 deployment traverses stairs up to 18 cm, gaps up to 70 cm, and steps up to 45 cm without manual filter calibration.

    Source: §I · p. 2

  • Takeaways & Limitations

    Ablations show denoising and contrastive alignment contribute at complementary reconstruction and representation levels and yield additive gains together.

    Source: §I · p. 2, §V · p. 8

Abstract

from arXiv · show

Vision-based legged locomotion methods assume clean depth at training time and rely on hand-tuned post-processing filters at deployment. However, filter parameters are rarely disclosed, hindering reproducibility, and performance degrades substantially when depth noise is left unaddressed. Building noise robustness directly into the learning pipeline would eliminate this dependency. While such robustness has been explored for proprioceptive inputs, analogous approaches for depth perception remain largely absent in legged locomotion. We propose DAWN (Denoising and Alignment in World models for Noise-robustness), a noise-robust perception framework for legged locomotion, which builds noise robustness directly into a world model via two modifications: (1) feeding noisy depth to the encoder while keeping clean depth as the reconstruction target, forcing the model to implicitly denoise its input; and (2) applying contrastive learning to align the latent states of noisy and clean depth. Importantly, DAWN is not tied to a specific noise model, requiring no manual tuning to the noise distribution at deployment. Furthermore, it incurs no additional inference cost over existing world model-based methods. Without any manual filter calibration -- relying solely on the learned noise-robust representation -- DAWN achieves zero-shot quadruped parkour on a Unitree Go1: traversing stairs up to 18 cm, clearing gaps up to 70 cm, and mounting steps up to 45 cm from raw depth observations. Ablation studies show that denoising and contrastive alignment contribute at complementary levels -- reconstruction and representation, respectively -- and yield additive gains when combined. Videos and code are available at: https://dawn-parkour.github.io/

I. INTRODUCTION

DAWN embeds depth-noise robustness in a world model rather than relying on deployment-time filters. It reports zero-shot Go1 parkour from raw depth and frames depth noise as a gap in prior locomotion approaches.

I. INTRODUCTION

  • Most vision-based locomotion methods train with clean depth and defer noise handling to deployment filters, whose parameters are seldom reported and vary with scene conditions.The D435i pipeline exposes six filter types with 12–24 interdependent parameters.Source: §I · p. 1

II. RELATED WORK

  • Prior depth-noise approaches use input-level augmentation or separate denoisers, while world-model denoising has addressed proprioception rather than spatially structured depth noise.Depth-noise statistics vary with scene geometry and illumination, unlike the low-dimensional, approximately i.i.d. proprioceptive noise considered in prior world-model work.Source: §I · p. 1

I. INTRODUCTION

  • DAWN adds noisy-input/clean-target reconstruction and contrastive alignment to an RSSM, embedding robustness in its latent space without deployment noise-model tuning or added inference cost.The method is trained end-to-end with the locomotion policy.Source: §I · pp. 1–2
  • 96.9% average success across stairs, gaps, and steps in simulation; zero-shot Go1 deployment traverses stairs up to 18 cm, gaps up to 70 cm, and steps up to 45 cm.The simulation result closes 77% of the gap between WMP and the clean-depth oracle; deployment requires no manual filter calibration.Source: §I · p. 2

B. Depth Sim-to-Real Transfer

Prior depth-transfer approaches use noise modeling, augmentation, filters, or separate modules, while depth-noise robustness has not been integrated into a world model. DAWN adds denoising reconstruction and contrastive alignment to an RSSM.

B. Depth Sim-to-Real Transfer

  • Input-level noise modeling and separate denoising modules can require scene-specific filters or auxiliary components, while repurposing a world model’s reconstruction objective to remove filter dependency had not been explored.Prior denoising methods and noise-synthesis approaches provide context for the depth-transfer problem, including auxiliary sensor fusion and physically informed noise models.Source: B. Depth Sim-to-Real Transfer · p. 2

C. World Models for Locomotion

  • World-model denoising had been applied to proprioception, but depth noise is spatially structured and varies with distance and depth discontinuities.The cited proprioceptive methods address noise that is low-dimensional and approximately i.i.d.Source: C. World Models for Locomotion · p. 2

D. Contrastive Learning for Locomotion

III. METHOD

  • DAWN modifies an RSSM with noisy-depth input and clean-depth reconstruction, plus a contrastive loss aligning latent states from noisy and clean observations.Fig. 1 highlights the denoising reconstruction loss and the contrastive loss applied to the projection head.Source: §III · p. 2, Figure 1 · p. 3

A. Preliminaries: World Model Learning

The WMP world model encodes observations into recurrent latent states, reconstructs observations, and trains a policy from the deterministic state. Its joint objective includes reconstruction and KL regularization.

A. Preliminaries: World Model Learning

  • At each timestep, the observation combines depth and proprioception, and depth is updated every k steps.The joint-position target action is denoted a_t.Source: A. Preliminaries: World Model Learning · p. 3
  • The RSSM uses a GRU-based sequence model for deterministic state h_t, a stochastic state z_t incorporating the observation, a prior state, and a reconstructed observation.The prior is predicted without observation access.Source: A. Preliminaries: World Model Learning · p. 3
  • The model jointly minimizes reconstruction and KL terms, while PPO trains the policy using deterministic state h_t as input.The reconstruction term retains information about the observation; the KL term regularizes the posterior toward the prior, with β as a hyperparameter.Source: A. Preliminaries: World Model Learning · p. 3
  • Standard WMP uses the same clean observation as encoder input and reconstruction target, and handles deployment depth noise with post-processing filters.DAWN changes the training procedure without changing the RSSM architecture.Source: A. Preliminaries: World Model Learning · p. 3

B. Denoising Reconstruction Objective

DAWN turns the RSSM reconstruction objective into denoising by feeding noisy depth to the encoder while retaining clean depth as the decoder target. Its information bottleneck and end-to-end encoder avoid a separate deployment filter stage.

B. Denoising Reconstruction Objective

  • Noisy depth replaces the clean encoder input while the clean observation remains the reconstruction target, requiring the latent state to support clean-depth recovery.This input–target mismatch makes the standard reconstruction loss a denoising objective.Source: B. Denoising Reconstruction Objective · p. 3
  • Because reconstructing clean depth does not require noise-specific information, the KL capacity constraint suppresses noise features and retains terrain-relevant geometry.The paper frames this behavior through the information bottleneck principle.Source: B. Denoising Reconstruction Objective · p. 3
  • WMP and DAWN retain identical RSSM architecture and policy training; their training configurations differ in encoder input and loss terms.Source: Table I · p. 4
  • The encoder learns a scene-conditioned nonlinear mapping end-to-end with the policy, replacing fixed-parameter filters that require scene-specific tuning.The denoising objective makes an additional deployment filter stage unnecessary.Source: B. Denoising Reconstruction Objective · p. 4

C. Contrastive Latent Alignment

Contrastive alignment complements denoising by directly aligning latent representations from clean and noisy depth. The loss operates on projected embeddings during training, while deployment removes the projection head.

C. Contrastive Latent Alignment

  • DAWN aligns clean and noisy encoder outputs at the same timestep, using a projection head to map posterior states into a contrastive embedding space.The denoising objective encourages noise-invariant reconstruction but does not explicitly constrain latent-space structure.Source: C. Contrastive Latent Alignment · p. 4
  • Embeddings from the same scene form positive pairs, while embeddings from different scenes form negative pairs in the NT-Xent loss.The batch contains N parallel environments; the loss uses cosine similarity, excludes the anchor from its comparison set, and applies temperature τ.Source: C. Contrastive Latent Alignment · p. 4
  • Applying contrastive loss to projected embeddings rather than z_t is intended to prevent encoder collapse and preserve information useful for policy learning.The projection head is removed at deployment; the loss acts only as a training-time regularizer, leaving inference identical to WMP.Source: C. Contrastive Latent Alignment · p. 4

D. Total Training Objective

DAWN combines denoising reconstruction and contrastive alignment in one training objective. The architecture and policy-training procedure remain unchanged; only the training signals are modified.

D. Total Training Objective

IV. EXPERIMENTAL RESULTS

The experimental setup evaluates simulated and zero-shot real-world locomotion under depth noise, using multiple terrain types and difficulty levels. The supplied results-section passages describe the evaluation design and baseline table scope, but provide no table values.

A. Experimental Setup

  • Simulation uses 4,096 parallel environments, 50 Hz control, and 64 × 64 depth images updated every five steps; results average three seeds and 100 episodes per condition.Source: A. Experimental Setup · p. 4
  • On the Unitree Go1, simulation-trained policies transfer zero-shot using raw depth with built-in camera filters disabled, across indoor and outdoor trials.Success rate is evaluated over 10 trials per difficulty level.Source: A. Experimental Setup · p. 4
  • The simulation uses a D435i-characteristic depth-noise model covering Gaussian sensor noise, edge dropout, and far-particle noise.Edge dropout has quadratic probability based on depth discontinuities; its probability is bounded by 0.60, and dropped pixels are filled by 3 × 3 max pooling.Source: A. Experimental Setup · p. 4
  • Evaluation spans slopes, stairs, gaps, and steps across five difficulty levels, with slopes included only in the baseline comparison.The stated ranges are 6–22° for slopes, 6–18 cm for stairs, 10–90 cm for gaps, and 10–54 cm for steps.Source: A. Experimental Setup · p. 4
  • Table II compares baseline methods under ×1.0 noise, reporting success rate and velocity tracking error averaged across difficulty levels.The caption specifies three seeds and 100 episodes.Source: Table II · p. 5
  • Compared methods include WMP with noise training and DAWN ablations that remove contrastive learning or RSSM denoising.The ablations isolate denoising-only and contrastive-only variants.Source: A. Experimental Setup · p. 5
  • Success rate measures terrain traversal without falling, while tracking error is mean squared error between commanded and measured velocity.Source: A. Experimental Setup · p. 5

B. Baseline Comparison

DAWN outperforms noise-trained and clean-depth baselines on the perceptive terrains, while baseline performance varies with how much each terrain depends on depth geometry. The comparison evaluates default ×1.0 noise across difficulty levels and seeds.

B. Baseline Comparison

  • The default ×1.0-noise baseline comparison averages results over all difficulty levels and three seeds to assess whether denoising belongs in the learning pipeline or can be deferred.Source: B. Baseline Comparison · p. 5
  • Blind performs near-perfectly on Slope but collapses on Gap and Step; WMP degrades most on Stair and Gap, where edge dropout obscures boundaries needed for geometry.Step is more forgiving because depth values on either side of its single discontinuity can reveal the height change.Source: B. Baseline Comparison · p. 5
  • EP trails WMP on every perceptive terrain, while WMP w/ N improves over clean-trained WMP but remains below DAWN.The comparison attributes EP’s weaker performance to less usable geometry reaching the policy through teacher–student distillation.Source: B. Baseline Comparison · p. 5
  • Across Stair/Gap/Step, DAWN reaches 96.9% average SR, closing 77% of the gap between WMP and the clean-depth Oracle.WMP w/ N recovers part of the gap but remains below DAWN.Source: B. Baseline Comparison · p. 5

C. Ablation Study

The ablation finds complementary gains from denoising and contrastive alignment, with their combination outperforming either component alone at the hardest difficulty. It also presents comparisons of simulated and real depth and real-world reconstructions.

C. Ablation Study

  • The ablation compares full DAWN, denoising-only and contrastive-only variants, and WMP baselines trained with noisy or clean depth; methods separate from level 4 onward.The hardest level is used to assess performance under the greatest noise stress.Source: C. Ablation Study · pp. 5–6
  • The figures compare simulated clean, noisy, and real D435i depth across three terrains, and show raw real depth above DAWN decoder outputs across sampled timesteps.Source: Figure 3 · p. 6, Figure 4 · p. 6
  • At the hardest level, DAWN reaches 88.2% SR; denoising alone adds 5.7 p.p. and contrastive alignment alone 4.6 p.p. over noise-only training, while both together add 11.5 p.p.The combined gain exceeds the sum of the individual gains.Source: C. Ablation Study · p. 6, Figure 2 · p. 6
  • Denoising steers the latent space toward noise-free geometry, while contrastive alignment regularizes encoder output so the decoder reconstructs from a cleaner state.The objectives provide additive performance gains when combined.Source: C. Ablation Study · p. 6

D. Noise Robustness Analysis

DAWN loses less success rate than the baselines as simulated noise rises beyond the training level. A latent-space visualization shows clean and noisy inputs overlapping within terrain clusters while distinct terrains remain separated.

D. Noise Robustness Analysis

  • From ×1.0 to ×2.0 noise at the hardest difficulty, DAWN loses 6.5 p.p. versus 17.5 p.p. for WMP, and its margin over WMP widens from 14.0 p.p. to 24.9 p.p.All policies were trained at ×1.0, making higher noise scales out-of-distribution.Source: D. Noise Robustness Analysis · p. 6
  • The real-world results figure reports SR for DAWN, WMP w/ N, and WMP across Stairs, Gap, and Step indoors and outdoors, over 10 trials per difficulty level.Source: Figure 5 · p. 7

E. Latent Space Analysis

  • In the t-SNE visualization, clean and noisy inputs overlap within each terrain cluster, while different terrain types form separated clusters.The passage interprets this pattern as noise-discarding representations that preserve terrain discriminability.Source: E. Latent Space Analysis · p. 7

F. Noise Model Validation

Across validation and deployment, DAWN reconstructs clean depth from real D435i input and performs strongly in outdoor parkour. The conclusion attributes gains to complementary denoising and contrastive alignment, while noting that the current noise model targets the D435i.

F. Noise Model Validation

  • Simulated noise visually matched real D435i depth, particularly missing pixels at depth discontinuities and far-particle artifacts, supporting the noise model’s capture of dominant sensor characteristics.Source: F. Noise Model Validation · p. 7

G. Real-World Depth Reconstruction

  • Real D435i depth inputs produced clean-depth reconstructions across consecutive Stair, Gap, and Step timesteps, indicating that the deterministic state discards sensor noise.Because the policy conditions on this state, it operates on noisy observations without external filters.Source: G. Real-World Depth Reconstruction · p. 7

H. Real-World Deployment

  • 80% (DAWN) vs. 70% (WMP w/ N) and 60% (WMP) on 18-cm Stair; Gap SRs were 60% vs. 40% and 30%, and Step SRs were 70% vs. 50% for both.The Gap and Step results are for 70-cm gaps and 45-cm steps, respectively.Source: H. Real-World Deployment · p. 7

V. CONCLUSION

  • Ablations indicate that denoising reshapes representations toward clean geometry while contrastive learning enforces noise-robust encoding, yielding a compound gain when combined.Source: §V · p. 8
  • The current noise model targets the D435i; extending it to other depth sensors is identified as future work.Source: §V · p. 8
Loading 2609.29092v1…