Source-linked AI summary
Robust Validation to Geometric Perturbations for Autonomous Pose Estimation
Gregoire Theau, Melanie Ducoffe
TL;DR
Safety-critical autonomous perception needs validation against physically plausible geometric perturbations, but first-order methods can miss pose-estimation vulnerabilities. The paper applies GLO to a YOLOv8-Pose and PnP pipeline, finding efficient search and operational failures beyond the safety limit while identifying scope boundaries for its guarantees.
Problem
Existing robustness validation emphasizes pixel-wise noise or single-stage classification, leaving geometric perturbations and downstream pose-estimation pipelines insufficiently covered.
Method
The paper evaluates GLO-based GeoRobust on an end-to-end YOLOv8-Pose and PnP Vision-Based Landing pipeline under geometric and photometric transformations.
Results
Over 80% of the worst-case transformation search space is pruned within ten iterations, while gradient-based APGD underperforms random sampling and GeoRobust exposes safety breaches.
Takeaways & Limitations
Global Lipschitzian optimization provides an efficient approach for isolating vulnerability zones before downstream formal verification in autonomous perception.
Takeaways & Limitations
The framework does not cover shifted transformations such as translation, and its theoretical assumptions can fail under candidate switching or degenerate PnP configurations.
Abstract
from arXiv · showhide
Deploying autonomous systems in safety-critical domains demands guaranteed robustness against physically plausible geometric perturbations rather than abstract pixel-wise noise. In vision-based navigation and autonomous landing, machine learning components require rigorous validation under dynamic operational conditions such as camera rotations and lighting shifts. Extending findings on the failure of first-order spatial attacks in classification, we show that standard gradient-based heuristics (e.g. APGD) similarly fail on for pose estimation, often performing worse than a simple random sampling baseline. To overcome these optimization bottlenecks, we reformulate pose estimation robustness within the framework of Global Lipschitzian Optimization (GLO). We argue that GLO offers a principled approach to robust validation, effectively localizing global optima with strong theoretical convergence guarantees. We evaluate this framework on a YOLOv8-Pose keypoint detector with a Perspective-n-Point (PnP) solver against rotation and contrast. In our evaluations, GLO successfully isolates critical failure modes where position deviations exceed safe operational limits, while rapidly pruning the search space by over 80%. To the best of our knowledge, this is the first study to extend geometric robustness validation to continuous keypoint regression and deep object detection, establishing a practical step toward certifying robust autonomous perception.
1 Introduction
Safety-critical autonomous perception requires robustness validation against physically plausible geometric disturbances and across multi-stage pipelines. This paper finds that first-order spatial search underestimates pose-estimation vulnerabilities and proposes Global Lipschitzian Optimization (GLO) as an alternative.
- Physically plausible transformations such as camera rotations and lighting variations better represent autonomous operating conditions than abstract pixel-wise noise.
- Existing geometric-robustness validation largely targets single-stage classification and does not cover downstream non-linear geometric solvers.
- First-order gradient search severely underestimates pose-estimation vulnerabilities and performs worse than naive random sampling.
- GLO reformulates pose-estimation robustness validation to address the limitations of first-order gradient search.
- Over 80% of the worst-case transformation search space is pruned within ten iterations using the GeoRobust implementation of GLO.
2 Spatial Robustness for Vision Based Landing
The Vision-Based Landing pipeline transforms aerial images, detects runway keypoints, selects a primary candidate, and estimates 3D aircraft position with PnP. Robustness is defined by whether the worst-case translation error remains below a safety threshold over bounded transformations.
- Vision-Based Landing estimates aircraft attitude relative to the runway through a multi-stage perception chain rather than isolated image classification.
- The transformation operator T applies bounded, physically plausible parameterized perturbations to input images.
- End-to-End Perception Pipeline: The detector outputs M candidate instances containing bounding boxes, confidence scores, and N 2D runway-keypoint coordinates.
- End-to-End Perception Pipeline: The selection operator chooses the highest-confidence candidate above τconf, whose 2D keypoints are passed to the PnP solver with known 3D runway coordinates.
- Bounded Pose Estimation Robustness Problem: Robustness requires maxθ∈Θ LE2E(θ) ≤ ϵsafe, while any configuration with LE2E(θ*) > ϵsafe constitutes a safety breach.
- Bounded Pose Estimation Robustness Problem: The evaluation uses ϵsafe = 1000 m and searches for the transformation maximizing end-to-end translation error.
3 Geometric Robustness for Pose Estimation
The paper contrasts gradient-based attacks with derivative-free global optimization for geometric pose-estimation robustness. Its convergence argument depends on continuity and non-degeneracy conditions that are challenged by candidate switching and ill-conditioned PnP configurations.
- Gradient-Based Baseline: APGD is adapted as a white-box baseline using BPnP to preserve gradient flow through the pose-estimation pipeline.
- Gradient-Based Baseline: The adapted APGD directly maximizes 3D translation error with respect to transformation parameters through an end-to-end chain-rule gradient.
- Global Lipschitzian Optimization: GeoRobust uses derivative-free DIRECT optimization and is compatible with any PnP algorithm because it evaluates the pipeline as a black box.
- Global Lipschitzian Optimization: GeoRobust partitions the bounded transformation domain into hyper-rectangles and retains potentially optimal regions that may contain the global optimum.
- Theoretical Guarantees: Under Lipschitzian assumptions, DIRECT guarantees asymptotic global-optimum convergence without gradient access.
- Theoretical Guarantees: Candidate switching creates discontinuities, while degenerate keypoint configurations can make the PnP solver locally ill-conditioned; shifted transformations such as translation are not covered.
- Theoretical Guarantees: The guarantee requires invariant candidate selection, globally Lipschitz keypoint extraction, and a non-degenerate continuously differentiable PnP solver.
4 Experimental Setup & Protocols
The evaluation benchmarks a YOLOv8-Pose/PnP vision-based landing pipeline under operationally motivated rotation and contrast perturbations. It measures pose deviation, survival, search-space reduction, active optimization regions, and critical perturbation thresholds across spatial and runtime budgets.
- Dataset and Pipeline: The benchmark uses LARDv2 aerial runway images spanning viewpoint, lighting, and weather variations.The dataset consists of runway images captured during approach and landing phases.
- Dataset and Pipeline: YOLOv8-Pose detects runway keypoints and regresses the four 2D corners required by the downstream PnP pose estimator.The model also predicts bounding boxes as part of the vision component.
- Perturbations: The perturbation space restricts evaluation to camera roll ϕ and contrast scaling α, modeling bank maneuvers, turbulence, and lighting shifts.The evaluation considers both one-dimensional roll and two-dimensional roll-plus-contrast settings, with reflection padding preventing black-border artifacts.
- Evaluation Protocols: Two protocols vary either rotation budgets of 2°–30° or per-image runtime budgets of 0.1–5.0 seconds.The rotation intervals represent operational conditions ranging from crosswinds and light turbulence to heavier turbulence and tight approach maneuvers.
- Metrics: The metrics include median end-to-end deviation error, survival rate, remaining search area, active potentially optimal subspaces, and critical perturbation thresholds.Survival rate measures samples maintaining pose estimation within the safety criterion, while remaining search area measures the normalized volume of candidate worst-case domains.
- Metrics: Critical perturbation thresholds identify the minimum perturbation magnitude that reduces pipeline survival below 10%.For one-dimensional evaluations, the threshold is specifically denoted ϕcrit.
5 Experimental Results and Robustness Analysis
The experiments evaluate pose-estimation robustness under camera rotation, contrast, runtime constraints, and approach-distance variation. GeoRobust consistently identifies stronger vulnerabilities than gradient-based APGD and rapidly narrows the search space.
- Solver Validation: 1,000 images validate that BPnP closely matches Iterative PnP, supporting its use as a differentiable surrogate for attack evaluation.With ground-truth keypoints, both achieve a 2.2% median relative translation error; noisy YOLOv8-Pose predictions also produce close median error profiles.
- 1D Spatial Robustness: Across ±2° to ±30° roll budgets, APGD fails to outperform Random Search, whereas GeoRobust achieves the lowest survival rate across all angular budgets.The results attribute APGD’s weakness to local minima in the highly non-convex transformation space and GeoRobust’s advantage to global Lipschitzian partitioning.
- Runtime Efficiency: Under a fixed ±15° rotation bound, runtime budgets from 0.1s to 5.0s expose differing APGD and GeoRobust time-to-failure behaviors using survival rate and median deviation error.Both methods can degrade system accuracy, but their convergence behaviors differ under strict computational constraints.
- 2D Perturbations: In the 2D rotation-contrast evaluation, GeoRobust drives survival rate to nearly 0% under extreme conditions, while APGD falls behind Random Search.The tested range expands from ±2.0° with contrast [0.9, 1.1] to ±30.0° with contrast [0.4, 2.5].
- 2D Runtime Evaluation: Under ±10.0° rotation and [0.7, 1.4] contrast bounds, GeoRobust crosses the 1000m failure threshold within 2.0s, while APGD plateaus around 500m.The comparison links APGD’s plateau to entrapment in local extrema and describes GeoRobust as navigating the non-convex 2D landscape efficiently.
- Distance-Based Vulnerability: Approach distance changes vulnerability: GeoRobust requires around 15° near 2600m and approximately 30° at mid-to-long range, while APGD often remains at the 45° experimental limit.The 45° boundary is defined as the limit for attacks that fail to breach the error threshold, and the analysis maps failure envelopes across approach phases.
- Convergence and Search Reduction: Within the first 10 iterations, GeoRobust reduces the bounding area enclosing worst-case transformations by over 80%, leaving less than 20% of the initial operational bounds.The convergence analysis tracks the Remaining Search Area and discards mathematically safe regions during global Lipschitzian search.
6 Conclusion and Future Work
The paper benchmarks end-to-end VBL robustness and finds that GeoRobust exposes severe spatial failures while sharply narrowing the search space. Future work targets temporal consistency, hybrid certification, and flight-certified architectures.
- Conclusion: The benchmark models an end-to-end VBL system that combines a YOLOv8-Pose detector with a PnP solver under physically plausible geometric and photometric perturbations.This establishes the evaluation setting for assessing autonomous vision-system resilience.
- Conclusion: GeoRobust exposed operational failures exceeding 1000m translation error within a 2.0s real-time budget.The comparison attributes this to GeoRobust navigating the non-convex transformation landscape more effectively than first-order methods.
- Conclusion: GeoRobust reduced the active search space enclosing worst-case perturbations by over 80% within 10 iterations.This isolates narrow vulnerability zones for subsequent computationally intensive exact verification.
- Future Work: Future work will evaluate temporal persistence, integrate formal verifiers into GeoRobust's bounded search spaces, and audit flight-certified target architectures.These directions cover sequential-frame consistency, hybrid certification, and deployment on certified hardware or software targets.