Source-linked AI summary
PASS: A Trustworthy Precision Approach Simulation System for CAT III Integrity Algorithm Development
Lang Zhu, Xin Zhang, Yulong Sun, Nan Liu, Jiahe Shen, Xiaodong Zhang, Xingqun Zhan
TL;DR
Reliable fixed-wing approach localization requires integrity as well as accurate estimates, while real flight tests make systematic data collection and failure reproduction difficult. PASS combines configurable approach simulation with continuous ground truth and integrity-support information, and SRST assesses category-level scene correspondence. The paper presents these as resources for developing and evaluating navigation and integrity algorithms, not as a navigation algorithm or a certified safety measure.
Problem
Method
Results
PASS and SRST provide bounded evidence about whether simulated approach scenes preserve the category-level organization of evaluated real scenes, rather than a universal realism measure.
Source: §2 · p. 6Takeaways & Limitations
The system provides controlled observations and traceable reference information for developing and evaluating navigation and integrity algorithms across approach conditions.
Source: §2 · p. 6PASS does not implement a particular navigation or integrity-monitoring algorithm, and SRST is not a universal visual-realism measure or certified safety threshold.
Source: §2 · p. 6
Abstract
from arXiv · showhide
For fixed-wing aircraft, reliable localization during approach and landing is critical to flight safety. Visual--inertial navigation is promising, but aircraft navigation also requires integrity: trustworthy outputs and timely warnings when they should not be used. Real approach data are costly to collect, while many visual errors and failures required for integrity algorithm development are difficult to reproduce safely. We present PASS, a trustworthy Precision Approach Simulation System for CAT III integrity algorithm development. In a unified temporal and geometric framework, PASS integrates configurable airport environments, prescribed trajectories, inertial simulation, calibrated stereo images, and continuous runway-relative ground truth. It supports navigation and integrity research through controllable sensor errors and environmental variations, computable visual projection errors, deterministic runway geometry priors, and condition-labeled data for statistical error modeling. We further propose Sim-to-Real Scene Trustworthiness (SRST), a measure based on DINOv2 feature distributions and UMAP, to quantify sim-to-real closeness. Experiments with representative stereo--inertial estimators demonstrate the usability of the generated sequences and quantify localization errors across algorithms and environments. PASS provides a controllable foundation for developing and evaluating visual--inertial navigation and integrity algorithms for fixed-wing precision approaches up to CAT III requirements.
1 Introduction
Fixed-wing approach research needs trustworthy, continuous visual–inertial data and integrity support that real flight tests cannot readily provide. PASS supplies these resources and introduces a stage-conditioned measure of sim-to-real scene correspondence.
1 Introduction
- Existing approach resources lack the controlled disturbances, independent visual-error references, runway priors, and condition labels needed to develop integrity algorithms reproducibly.Real approaches are costly and constrain systematic reproduction of adverse conditions and rare failures.Source: §1 · p. 2
- PASS combines prescribed trajectories, inertial simulation, configurable airport environments, calibrated stereo imaging, and traceable runway-relative ground truth in a shared temporal and geometric framework.PASS provides data and interfaces for downstream navigation and integrity work; it does not itself implement those algorithms or output a protection level.Source: §1 · p. 3
- PASS supports integrity research with controllable errors and environmental variation, ground-truth-referenced visual projection errors, deterministic runway priors, and condition-labeled data for statistical modeling.These resources support error modeling and integrity analysis across conditions while leaving downstream algorithms independent of the simulator.Source: §1 · p. 3
- SRST compares simulated and real approach scenes using frozen DINOv2 features and stage-specific joint UMAP representations, without requiring image-level pairs.The measure assesses category-relative scene correspondence across coastal, mountainous, desert, and urban airports.Source: §1 · p. 3
- The paper describes the dataset as planned for public release upon publication.Source: §1 · p. 4
2 Related Work
Existing runway datasets and general-purpose simulators provide useful imagery or navigation data, but do not jointly meet the needs of continuous approach navigation and integrity research. PASS combines approach-oriented data and integrity support with a bounded, stage-conditioned assessment of scene correspondence.
2 Related Work
- Runway-image datasets support perception, but their primarily image-level annotations do not provide continuous calibrated stereo–inertial measurements, runway-relative states, or time-aligned error references.As a result, they cannot independently support continuous visual–inertial navigation or integrity analysis throughout an approach.Source: §2 · p. 4
- General-purpose simulation resources provide repeatable data, but lack an integrated approach-oriented framework for runway geometry, runway-relative states, calibrated sensing, controllable inconsistencies, and truth-referenced visual errors.Adapting them for systematic visual-error modeling and integrity studies would require substantial redevelopment.Source: §2 · pp. 4–5
- Aggregate feature-distribution metrics compare unpaired images, but depend on the feature extractor and sample choices and may not reveal category coverage or scene-specific correspondence.The reviewed approaches also differ from conditional, spatial, and task-level evaluation, which answer distinct questions about simulation credibility.Source: §2 · p. 5
- The comparison concerns runway-oriented datasets and synthetic resources for fixed-wing precision-approach navigation and integrity research.Source: Table 1 · p. 6
- The comparison distinguishes implemented-but-not-integrated support from released condition diversity: “Supported” is not an integrated runway-oriented data product, while “Preset” is not user-controlled error excitation.Source: §2 · p. 6
- PASS addresses the identified gaps with integrated approach data and integrity support, while SRST provides bounded evidence about category-level scene organization rather than universal realism or a certified safety threshold.PASS does not implement a particular navigation or integrity-monitoring algorithm.Source: §2 · p. 6
3 PASS: A Trustworthy Precision Approach Simulation System for CAT III Integrity Algorithm Development
PASS is presented as a framework for visual–inertial navigation and CAT III integrity algorithm development. Its architecture connects scenario definition, trajectory and inertial simulation, and stereo imaging, alongside a stage-conditioned assessment of scene trustworthiness.
3 PASS: A Trustworthy Precision Approach Simulation System for CAT III Integrity Algorithm Development
- PASS is designed to support visual–inertial navigation and CAT III integrity algorithm development through a unified approach simulation system.Source: §3 · p. 6
- The system connects scenario definition, trajectory and IMU generation, and synchronized stereo imaging through a shared time base and coordinate transformations.Source: Figure 1 · p. 7
- The paper also introduces a sim-to-real quality assessment based on image distributions conditioned on approach stage in a high-dimensional feature space.Source: §3 · p. 7
3.1 System Architecture
PASS generates synchronized stereo imagery and inertial measurements from prescribed aircraft motion, linking the modules through shared timing and coordinate transformations. The resulting sequence structure combines sensor data with ground truth, calibration, and runway references.
3.1 System Architecture
- A shared prescribed trajectory drives both navigation-state and inertial-measurement generation and the virtual motion replayed for stereo images.The common time base and explicit coordinate chain maintain temporal and geometric consistency across the modules.Source: §3.1 · p. 7
- PASS sequences combine integrity-support conditions and runway priors with continuous trajectory and runway-relative ground truth, inertial measurements, synchronized imagery, and calibration and frame parameters.Source: §3.1 · p. 7
- Trajectory files serve as both released ground truth and the motion input for image generation, so visual motion is not generated independently of the IMU and pose records.Source: §3.1 · p. 7
- Table 2 presents the inputs and outputs of PASS’s three modules.Source: Table 2 · p. 8
- The coordinate framework links geodetic and local navigation frames with runway, aircraft body/IMU, stereo-camera, and image frames.Source: Figure 2 · p. 8
3.2 Modular Subsystems
PASS coordinates its visual environment, prescribed motion, inertial simulation, and stereo imaging through a shared framework. Its modules keep generated observations and geometric references consistent while allowing operating conditions and sensor characteristics to be configured.
3.2 Modular Subsystems
- The three coordinated subsystems allow the visual environment, trajectory, sensor configuration, and geometric references to vary independently while preserving a unified data-generation interface and cross-modal consistency.A shared coordinate framework connects trajectory, inertial, visual, and runway-referenced information.Source: §3.2 · p. 8
Airport and runway construction.
- Table 3 summarizes the parameterization of the PSINS-based IMU simulation model.Source: Table 3 · p. 9
Controlled visual conditions.
- Illumination and atmospheric conditions can vary individually or together while trajectory, camera calibration, and runway geometry remain fixed.This separates environmental changes from unintended changes in aircraft motion or sensor geometry.Source: Controlled visual conditions · p. 9
3.2.2 Trajectory and IMU Module
- PSINS propagates prescribed approach boundary and motion conditions into a continuous trajectory used for both inertial-data generation and visual replay.The shared motion basis keeps simulated inertial measurements, rendered motion, and runway-relative pose ground truth consistent.Source: §3.2.2 · p. 9
- Configurable IMU errors include bias, random walk, correlated bias, scale factor, and inter-axis misalignment, without imposing an additional distributional assumption.The error parameters are applied to sampled inertial increments.Source: §3.2.2 · pp. 9–10
3.2.3 Stereo Imaging Module
- The fixed stereo-camera model is calibrated from multi-view checkerboard observations, with a mean reprojection error of approximately 0.54 pixels.The calibration geometry and residual distribution are shown in Fig. 3.Source: §3.2.3 · p. 10, Figure 3 · p. 11
3.3 Support for Navigation Integrity (CAT III)
PASS organizes its integrity support around controllable perturbations, independent truth and geometry, and condition-diverse observations. Its environmental variations can change visibility while leaving the prescribed trajectory and runway reference geometry unchanged.
3.3 Support for Navigation Integrity (CAT III)
- Integrity concerns trust in a navigation solution and timely alerts for hazardous misleading information, requiring more than an estimator’s sensor measurements.Source: §3.3 · p. 11
- Environmental variations in illumination and weather leave the prescribed aircraft trajectory and runway reference geometry unchanged.Source: Figure 4 · p. 12
- PASS groups integrity support into controllable perturbation and fault injection, independent truth and runway geometry, and condition-diverse data for statistical error modeling.Source: §3.3 · pp. 11–12
A. Controllable scenario perturbation and fault injection
PASS separates normal visual-condition changes from explicit faults so their sources and configurations can be controlled. The approach trajectory and reference geometry remain fixed during visual-condition perturbations.
A. Controllable scenario perturbation and fault injection
- Illumination, weather, runway appearance, dynamic objects, and partial occlusion vary while trajectory, nominal camera geometry, and runway frame remain unchanged.This design associates visibility and feature-localization changes with known operating conditions.Source: A. Controllable scenario perturbation… · p. 12
- Changing the rendered runway and its geometric description together creates a configuration change; perturbing the stored prior alone creates a controlled map-prior fault.Camera calibration or installation parameters can also be varied independently to represent sensor-model inconsistencies.Source: A. Controllable scenario perturbation… · p. 12
B. Independent truth and runway geometry references
PASS provides estimator-independent trajectory truth and deterministic runway geometry for evaluating visual measurements. Projecting runway references into images makes truth-referenced visual errors computable across an approach.
B. Independent truth and runway geometry references
- PASS supplies continuous six-degree-of-freedom aircraft trajectory truth in the runway frame, synchronized with stereo images and independent of the evaluated estimator.The trajectory and calibrated camera model determine camera pose at each image timestamp.Source: B. Independent truth and runway… · pp. 12–13
- Birds create localized motion and partial runway-feature occlusion without changing the prescribed trajectory or reference geometry.Source: Figure 5 · p. 13
- The runway frame, dimensions, bearing, corners, and principal line endpoints provide deterministic geometry rather than detector-dependent image annotations.Fig. 6 shows the runway-centered reference frame and geometric features.Source: B. Independent truth and runway… · p. 13, Figure 6 · p. 13
- Projecting runway references with ground-truth pose and calibrated camera parameters enables comparison with detected corners or lines to compute visual projection errors.For line features, signed point-to-line distances can be calculated repeatedly to produce an error time series; detector and integrity-method choices remain downstream.Source: B. Independent truth and runway… · pp. 13–14
C. Physics-diverse data for statistical error modeling
PASS produces condition-labeled approach data for estimating stage- and condition-dependent visual error distributions, and SRST assesses simulated imagery through category-relative correspondence with real images. The SRST assessment uses separate approach stages and alternative category resolutions, but sparse, unpaired real imagery limits how distributional discrepancies can be interpreted as scene-level correspondence.
C. Physics-diverse data for statistical error modeling
- Condition-labeled sequences across airports, runway configurations, weather, visibility, occlusion, and approach geometry support estimating visual error distributions and testing their applicability across operating conditions.These samples can inform dynamic measurement covariances and conservative error envelopes, while multi-airport data help distinguish recurring behavior from airport-specific effects.Source: C. Physics-diverse data for… · p. 14
- PASS’s three information categories link controlled disturbance generation, independent error evaluation, and condition-dependent statistical characterization without prescribing a navigation estimator or integrity architecture.The simulation data alone do not establish certified CAT III integrity performance or replace validation with representative real-flight observations.Source: C. Physics-diverse data for… · p. 14
3.4 Sim-to-Real Scene Trustworthiness inspired by Human Cognition
- Sparse real-image sampling and unknown camera geometry, aircraft state, and approach conditions limit interpretation of a single feature-distribution discrepancy as evidence of scene-level correspondence.A score above 0.90 is an operational criterion for favorable category-relative correspondence in the evaluated images, not an aviation safety threshold.Source: §3.4 · pp. 15, 18
- SRST measures category-relative correspondence within each approach stage, comparing four representative environments and analyzing middle and low approach images separately.The assessment avoids requiring exact real-image counterparts and also considers a three-category taxonomy that merges coastal and urban scenes.Source: §3.4 · p. 15
- SRST jointly embeds real and simulated image descriptors with UMAP, then tests whether simulated images favor their assigned scene category over plausible alternatives.The 4c criterion requires a category to prevail over at least two alternatives; 3c merges coastal and urban scenes and requires it to prevail over both remaining alternatives.Source: §3.4 · pp. 16–17
4 Experiments
The experiments evaluate PASS sequence usability with stereo–inertial estimators and assess sim-to-real scene correspondence using SRST. They report estimator- and environment-dependent errors, alongside category-relative correspondence that varies with scene taxonomy and approach stage.
4.1 Functional Usability of PASS Sequences
- The evaluation used 72 runs across three airports, two estimators, and four windows, with one-time ideal initialization and no later ground-truth updates.Each airport–estimator–window combination was run three times; longer windows combine earlier approach stages with longer estimation intervals.Source: §4.1 · p. 19
- VINS-Fusion completed every run, while ORB-SLAM3 completed all but one 30 s KSMO run; both produced evaluable trajectories across three airports.The interrupted ORB-SLAM3 run terminated during backend bundle adjustment.Source: §4.1 · p. 19
- At 30 s, mean terminal position errors over completed repetitions were 19.95, 40.51, and 23.74 m for VINS-Fusion at ZSPD, KSMO, and LGMK, respectively, and 3.02, 18.51, and 5.48 m for ORB-SLAM3.Source: §4.1 · p. 19, Table 6 · p. 20
- Errors rose with longer windows at every airport, but their magnitude depended on airport and estimator; longer windows also began earlier in approach.The trend therefore cannot be attributed to duration or altitude alone; repetitions for completed conditions remained closely grouped.Source: §4.1 · p. 20, Figure 9 · p. 21
4.2 Sim-to-Real trustworthiness of PASS Sequences
- The four-category UMAP showed separated middle-stage mountainous and desert regions, while coastal and urban scenes neighbored each other and low-stage categories overlapped more.The proposed runway-occupancy explanation was not measured.Source: Figure 10 · p. 22, §4.2 · p. 23
- In five reported four-category realizations, 90.1–94.8% of simulated images met the criterion of favoring their intended real-image category over at least two alternatives.All middle-stage desert and mountainous images and all four low-stage groups passed in those realizations; remaining discrepancies clustered in middle-stage coastal and urban categories.Source: §4.2 · p. 22, Table 8 · p. 23
- The exploratory three-category assessment excluded 61 mixed arid–coastal simulations; merging urban and coastal scenes yielded more coherent category correspondence, not evidence of closer feature points.The broader taxonomy addresses ambiguity in the scene labels; its illustrative realization is shown in Fig. 11.Source: §4.2 · pp. 24–25, Figure 11 · p. 25
- QSRST averaged 98.08% across 50 UMAP initializations (SD 2.72 percentage points; range 90.55–100%), and all exceeded 90%.The three-category assessment scored 444 real reference images and 307 simulated images.Source: §4.2 · pp. 23–24, Table 9 · p. 24
5 Conclusion
PASS is a synthetic stereo–inertial dataset and extensible simulation framework for fixed-wing runway approach navigation and integrity research. Its current release primarily represents prescribed straight-in approaches.
5 Conclusion
- PASS provides synchronized stereo images and IMU measurements, continuous six-degree-of-freedom ground truth, calibration and frame metadata, and runway geometry and relative states.The conclusion describes these outputs as reference information for characterizing navigation errors and investigating integrity monitoring.Source: §5 · p. 25
- The current release primarily represents prescribed straight-in approaches; future work will extend it to path and flight-state deviations, touchdown, and ground roll.The planned extensions are to retain calibration and coordinate-frame consistency.Source: §5 · p. 25