Source-linked AI summary
Adapting a Foundation Model for Lunar Surface Height Estimation
Patrick Bauer, Marius Schwinning, Melanie Siegel, Andreas Weinmann, Hichem Snoussi
TL;DR
The paper addresses the need for lunar surface-height information beyond 2D imagery and the lack of target-domain adaptation for DAV2. It fine-tunes DAV2 on publicly available SPG-derived DEMs, improving performance and producing a reliable relative lunar-height estimator.
Problem
Image-only lunar hazard detection omits surface height, while DAV2 has been used for lunar DEM estimation without adaptation to the target domain.
Method
The approach fine-tunes the DAV2 foundation model on publicly available, high-resolution SPG-derived lunar DEM products using LoRA and selected unfrozen decoder layers.
Results
Fine-tuned DEM-DAV2 shows improved qualitative and quantitative test-set performance compared with zero-shot DAV2.
Takeaways & Limitations
DAV2 can be transformed into a reliable relative depth estimator for the lunar surface through lunar-data fine-tuning.
Abstract
from arXiv · showhide
Digital elevation models (DEMs) can provide accurate height information, making it invaluable for analyzing the lunar surface. As the European Space Agency (ESA) prepares for future lunar missions that aim to land on the Moon, a precise method for height estimation will be essential for hazardous terrain that could endanger the landing approach. Traditional approaches to generate DEMs from imagery, such as shape from shading (SfS) and stereophotogrammetry (SPG) have been proven highly valuable for this task. However, due to advancements in machine learning, especially computer vision, the focus has shifted towards monocular depth estimation via deep learning. The lunar surface is covered by rocks and craters, and classic hazard detection methods rely solely on 2D image data. Our goal is to address this issue by developing a relative lunar surface height estimator that can provide additional information for hazard localization. In this letter, we present a methodology that builds on the well-known zero-shot relative depth estimation model Depth Anything V2 (DAV2). Other works have been using it as a state-of-the-art comparison for their proposed lunar DEM estimation method, but without adaptations to the target domain. Thus, it may underperform. Therefore, we propose a fine-tuning strategy with publicly available SPG-derived DEM data of the lunar surface. Our results demonstrate a significant improvement in performance compared to the zero-shot model, effectively transforming DAV2 into a reliable relative depth estimator of the lunar surface.
1 Introduction
Lunar landing and hazard-localization efforts need surface-height information because image-only detection omits the third dimension. The paper therefore adapts DAV2 to estimate relative lunar surface height using high-resolution SPG-derived DEMs.
- Lunar landing is challenging because the surface contains craters and boulders with diverse sizes and shapes.
- Image-based crater detectors typically omit the third dimension, which can reduce detection quality for degraded craters.
- SPG can generate lunar DEMs at up to 2 m/px, but stereo observations covering the same scene are limited.
- Monocular depth models estimate depth from a single image, and foundation models such as DAV2 support downstream computer-vision tasks after extensive pre-training.
- The proposed approach fine-tunes DAV2 with publicly available, high-resolution SPG-derived DEMs to estimate relative rather than absolute lunar surface height.
2 Data and Methodology
The methodology adapts DAV2 for lunar relative-height estimation using SPG-derived DEMs, parameter-efficient fine-tuning, selective decoder updates, and a hybrid loss.
- Lunar elevation data: SPG-derived DEMs provide lunar elevation maps at resolutions up to 2 m/px, but stereo pairs cover only around 5% of the surface.Laser-altimeter products provide accurate height maps at coarse resolution, whereas SPG depends on repeated observations from oblique angles.
- Depth Anything V2: DAV2 estimates relative inverse-depth maps from single images after processing them through a DINOv2-based Transformer encoder and DPT decoder.The ViT-L variant is used, with 24 Transformer blocks in the encoder and multistage feature fusion in the decoder.
- Parameter-efficient fine-tuning: The model is fine-tuned with LoRA applied to the encoder’s query and value matrices at rank r = 8.LoRA introduces low-rank matrices into pretrained high-dimensional weight matrices without adding inference latency.
- Parameter-efficient fine-tuning: Three of four RefineNet blocks and the output head are unfrozen while the Reassemble layers remain frozen, fine-tuning approximately 8 million of 335 million parameters.This selectively reduces the imbalance between learnable encoder and decoder parameters.
- Hybrid training objective: Training uses a hybrid loss combining Berhu, multiscale gradient-matching, and surface-normal terms after scale-and-shift-invariant normalization.The gradient term uses four scales with halved image resolution at each level; the normal term is scaled with j = 30, using α = 1.0, β = 2.0, and γ = 0.5.
- Training procedure: Data augmentation flips images vertically and horizontally with probability 0.5, and training runs for 20 epochs on four H100 GPUs with batch size 8 per GPU.The procedure also uses AdamW, weight decay of 1 × 10^-2, and linear warmup for the first 10% of iterations.
3 Results
DEM-DAV2 improves lunar relative height estimation over zero-shot DAV2 in qualitative and quantitative test-set evaluations. The fine-tuned model better captures both local features and overall terrain relief.
- Qualitative Evaluation: DEM-DAV2 captures high-frequency features and overall relief more effectively than DAV2 in visual test examples.DAV2 recognizes local features such as craters but misses broader structural elevation differences.
- Quantitative Evaluation: The quantitative test-set evaluation reports higher metrics for DEM-DAV2 than for DAV2.MAE and RMSE are averaged over the full test dataset after per-tile inference.
- Quantitative Evaluation: Fine-tuning the pre-trained DAV2 model for DEM estimation outperforms using it without modification.The improved metrics support the qualitative comparison between DEM-DAV2 and zero-shot DAV2.
- Qualitative Evaluation: In both analyzed 259×259 images, DEM-DAV2 produces height profiles superior to DAV2.DAV2 performs well in one example but struggles with the high-frequency profile in the other.
4 Conclusion and Future Work
The paper adapts DAV2 for lunar DEM height estimation using LoRA and selected decoder-layer unfreezing, trained and evaluated on SPG-derived DEM products. The resulting model is a reliable relative lunar height estimator, but it cannot provide real-time or absolute-height DEMs and remains lunar-specific.
- Conclusion and Future Work: LoRA parameters and selected unfrozen DPT decoder layers address the domain gap when adapting DAV2 to lunar remote sensing.The strategy uses publicly available SPG-derived DEM products for training and evaluation.
- Conclusion and Future Work: The adapted DAV2 becomes a reliable relative height estimator for the lunar surface.The paper frames this as transforming the foundation model for lunar DEM estimation.
- Conclusion and Future Work: DAV2 cannot provide real-time DEM estimation on a CPU because its inference time is relatively long.
- Conclusion and Future Work: The training pipeline does not reconstruct absolute heights and the approach is applicable only to the lunar surface.Adapting it to bodies such as Mars would require new training because surface characteristics differ.