Efficient and High-Quality Depth Estimation via Pixel-Space Diffusion with Linear Attention

Motivation

• High-resolution depth estimation exposes a fidelity–efficiency trade-off because generative models face quadratic attention costs, multi-step denoising, or information loss and weak global consistency.Latent-space models lose information through the VAE bottleneck, while pixel-space models with standard attention are expensive at high resolutions.Source: §1 · pp. 2–3

Method

• Lapis introduces a Patch-level Consistency Module that integrates semantic and spatial priors to restore global structural coherence.The module addresses patch-level structural misalignment in linearized attention.Source: §1 · pp. 3–4
• The Pixel-level Refinement Module uses skip-connection-based pixel correspondence to recover sharp boundaries and fine-grained geometric details.It re-integrates pixel-level input cues during decoding to counteract detail loss.Source: §1 · pp. 3–4
• Direct pixel-space x-prediction targets the clean data manifold and suppresses the noise and instability of one-step diffusion.This replaces volatile v-prediction under the manifold assumption that natural data lie on a low-dimensional manifold.Source: §1 · pp. 3–4

Results

• At 1080P, Lapis achieves a 5.1 average AbsRel, a 10% relative improvement over the strongest competitor, while requiring only 13% of its latency.Across 12 benchmarks, it reports SOTA accuracy and boundary sharpness across varying resolutions with competitive efficiency.Source: §1 · pp. 3–4

Monocular Depth Estimation

• Discriminative monocular depth models progressed from domain-specific CNN regression toward open-domain systems using diverse datasets, Vision Transformers, pre-trained encoders, and massive training data.Recent discriminative models achieve strong zero-shot robustness but may produce over-smoothed results.Source: §2.1 · p. 4

Generative Depth Estimation

• Early generative depth models use latent diffusion and ϵ-prediction with 10–50 sampling steps, while flow-matching v-prediction methods still require at least 4 steps to suppress noise artifacts.Existing one-step x-prediction models form another acceleration direction.Source: §2.2 · p. 5

Linear Attention

• Linear attention removes Softmax through kernel approximations to achieve O(N) complexity, but dense depth estimation is especially sensitive to representational decay from linear approximations.Prior visual applications mainly target categorical tasks or perceptual plausibility rather than metric accuracy.Source: §2.3 · p. 5

3.1 One-step Pixel-Space Diffusion

• Lapis replaces standard DiT self-attention with linearized attention and directly predicts clean depth maps in pixel-space using one-step diffusion.The framework avoids latent autoencoders and multi-step sampling overhead.Source: §3 · p. 5, §3.1 · pp. 6–7, Linear attention formulation · p. 7

3.3 Patch-level Consistency Module

• The Patch-level Consistency Module integrates semantic and spatial priors to enforce global structural alignment across the token grid.Global and local consistency mechanisms address structural misalignment and patch discontinuities.Source: Figure 3 · p. 6, §3 · p. 6, The fidelity gap in linearization · p. 8

3.1 One-step Pixel-Space Diffusion

• The training objective combines velocity-based flow supervision with multi-scale gradient matching in x-space to balance accuracy and boundary sharpness.The total loss is L_total = L_velocity + λ_grad L_grad, with λ_grad controlling the trade-off.Source: Training losses · p. 7

3.3 Patch-level Consistency Module

• The ConvFFN adds a residual 3 × 3 depth-wise convolution to preserve spatial continuity across adjacent tokens and suppress grid artifacts.Its local inductive bias complements the global structural rectification provided by the PCM.Source: Local spatial continuity · p. 9

3.4 Pixel-Level Refinement Module

• The Pixel-level Refinement Module reconnects pixel-level input cues with upsampled latent features to recover sharp geometric boundaries and fine details.The PRM uses long-range skip connections, positional encoding, projection, and PixelShuffle upsampling before depth prediction.Source: §3.4 · pp. 9–10

Implementation Details

• About 122K synthetic training samples from five datasets are resized to 1024 × 768 and concatenated without weighted sampling.The datasets are Hypersim, TartanAir, VKITTI2, UrbanSyn, and MatrixCity.Source: Training datasets · p. 11
• 520M parameters and a frozen DINOv2-Large semantic encoder define the main architecture, while the PRM uses three blocks.The backbone preserves DiT-Large hidden dimensions and block count while replacing attention with linearized operations.Source: Model architecture details · p. 11

Native Resolution

• Lapis achieves SOTA zero-shot relative depth performance on five native-resolution real-world datasets.Evaluation uses AbsRel and δ1 accuracy on NYUv2, KITTI, ETH3D, ScanNet, and DIODE.Source: Native resolution · p. 12

Test-time Resolution Scaling

• Approximately 10% and 19% AbsRel improvements are reported at 1080P and 1440P, respectively, over current SOTA models.These evaluations use five high-resolution datasets and assess preservation of global structure under resolution changes.Source: Test-time resolution scaling at 1080P… · pp. 12–13

Boundary Sharpness

• Lapis consistently outperforms discriminative and generative baselines in boundary-sharpness F1-score across test resolutions.The metric targets sharp occlusion boundaries and fine-grained structural transitions on Hypersim, Sintel, and Spring.Source: §4.3 · p. 13, Table 3 · p. 13

Ablation and Prediction Targets

• One-step x-prediction is the optimal sampling choice, while PCM supports global coherence and PRM restores high-frequency geometry.Removing Global PCM sharply reduces accuracy, Local PCM raises AbsRel from 4.8 to 5.1, and PRM is essential for detailed geometry despite marginal accuracy effects.Source: Prediction targets and diffusion steps · pp. 14–15, Effectiveness of components · p. 14

Conclusion

• Lapis decouples patch-level context modeling from pixel-level refinement and uses direct x-prediction to address linearized-backbone degradation and one-step diffusion noise.The model is designed for efficient high-resolution depth estimation.Source: §5 · p. 15
• Extensive experiments show SOTA zero-shot performance while significantly reducing inference latency.The paper positions Lapis as a scalable computational foundation for real-world 3D perception.Source: §5 · p. 15

Architecture

• Lapis and its ablation variant remain comparable with DiT-L and DiT-B in hidden dimensions and block counts.Table F lists the detailed architectural configurations.Source: A More Implementation Details · p. 21

Relative Depth Evaluation

• Relative-depth evaluation follows Marigold preprocessing, least-squares alignment, and metric computation protocols.ETH3D is downsampled to 2048 × 1360, and unreliable DIODE transitions are masked using a Sobel threshold of 0.3.Source: §B.1 · p. 21

Boundary Sharpness Evaluation

• Boundary-sharpness evaluation follows the Depth Pro protocol with least-squares alignment and clipping to each dataset’s valid depth range.Predictions are resized to ground-truth resolution before final sharpness metrics are computed.Source: §B.2 · p. 21

Runtime

• Runtime is measured on one NVIDIA RTX Pro 6000 Blackwell GPU with batch size 1.Results average 100 iterations after 20 warm-up rounds, with compilation and memory-efficient attention used to reduce overhead.Source: §B.3 · p. 21
• Lapis significantly outperforms existing generative models in inference latency while achieving throughput competitive with regression-based architectures.Table 4 reports latency, throughput, and peak GPU memory across resolutions.Source: §B.3 · p. 21

Qualitative Point Clouds

• Lapis suppresses flying-pixel artifacts and produces more spatially consistent surfaces than compared discriminative and latent-space generative methods.The qualitative comparison focuses on point clouds without post-processing and highlights artifacts in red boxes.Source: §B.3 · p. 22

• Edge-aware point clouds evaluate Lapis from the perspective of 3D structural integrity.The evaluation is conducted on the official Hypersim test split.Source: C Edge-Aware Point Cloud Evaluation · p. 22
• The evaluation is presented as more rigorous than boundary F1-score for assessing structural fidelity.Source: C Edge-Aware Point Cloud Evaluation · p. 22
• The setup addresses flying pixel artifacts associated with over-smoothing or lossy VAE compression in other paradigms.Source: C Edge-Aware Point Cloud Evaluation · p. 22

Kernel Ablation

• Linearized configurations maintain performance parity with standard Softmax attention, while the ELU + 1 kernel achieves a slightly superior overall result.Source: D Extended Ablation Study · p. 23, §D.1 · p. 23

Semantic Conditioning Strategies

• Direct one-to-one spatial correspondence in MLP and AdaLN variants significantly outperforms global Linear Cross-Attention for preserving geometric layout.Source: Table I · p. 23, §D.2 · p. 23

Kernel Ablation

• The kernel ablation compares ReLU, ELU+1, Focused ReLU, and standard Softmax attention within the complete Lapis architecture.Source: §D.1 · p. 23

Qualitative Results

• Qualitative comparisons on Hypersim, Sintel, and Spring show sharp structural boundaries across complex indoor and wide-scale outdoor scenes.Source: §D.2 · p. 24

Semantic Conditioning Strategies

• Interleaved AdaLN-Zero is selected as the final semantic-conditioning configuration because it offers slightly better average results without additional parameters.Source: §D.2 · p. 24

• Lapis preserves structural fidelity across Hypersim, Sintel, Spring, and open-domain images from Pexels and Unsplash.Source: E More Visual Results · p. 24
• The visual comparisons are intended to demonstrate robustness across diverse scenarios.Source: E More Visual Results · p. 24

• Linear attention currently exhibits numerical instability in reduced-precision formats such as fp16.Source: F Limitations and Future Work · p. 24
• The instability is attributed to the lack of explicit normalization in attention-score computation.Source: F Limitations and Future Work · p. 24
• Lapis currently focuses on single-image relative depth estimation rather than metric depth or video-consistent estimation.Source: F Limitations and Future Work · p. 24
• Future extensions to metric depth and video consistency would broaden applications in robotics and temporally coherent scene reconstruction.Source: F Limitations and Future Work · p. 24