Source-linked AI summary
SeA-RVINS: Semantic-Aware Tightly Coupled RTK-Visual-Inertial System with Correlation-Preserving Robust Estimation for Urban Navigation
Wang Hu, Bo Wu
TL;DR
Urban localization can be undermined when corrupted GNSS measurements or false visual associations persist in tightly coupled estimators. SeA-RVINS combines semantic-aware visual screening, hybrid ambiguity continuation, and correlation-preserving robust GNSS estimation. On an approximately 20-km TEX-CUP route, its latent-pivot configuration achieved 100% availability and a 1.60-m maximum horizontal error.
Problem
Corrupted GNSS factors, false visual landmarks, and wrongly continued ambiguities can persist through states and marginalization in tightly coupled estimators.
Source: §I · p. 1Method
SeA-RVINS combines a semantic-aware learned stereo frontend, hybrid ambiguity continuation, and correlation-preserving robust GNSS estimation.
Source: §V · p. 7Results
On the approximately 20-km TEX-CUP route, the latent-pivot configuration achieved 100% availability and a 1.60-m maximum horizontal error.
Source: §V · p. 8Takeaways & Limitations
The reported route results show robustness beyond average positioning accuracy, with low maximum errors and full availability.
Source: B. Results and Discussion · p. 7The authors identify partial IAR with dedicated robustness validation as future work, alongside lane-map aiding for long GNSS-blocked segments.
Source: §V · p. 8
Abstract
from arXiv · showhide
Reliable absolute pose estimation in urban environments is undermined by outlier measurements and incorrect temporal associations that can persist in tightly coupled estimators. Global Navigation Satellite System (GNSS) observations provide globally referenced measurements but are prone to multipath effects. Visual-inertial sensing supplies local motion constraints, but false visual associations can corrupt the estimator. We present SeA-RVINS, a fixed-lag factor-graph Real-Time Kinematic (RTK) visual-inertial system for robust urban pose estimation. A semantic-aware learned stereo frontend rejects unreliable tracks before persistent landmarks enter the graph. For double-differenced GNSS measurements, SeA-RVINS applies Dynamic Covariance Scaling through configurable batch, scalar, and latent-pivot robust formulations while retaining the shared-pivot correlation structure. We propose a hybrid ambiguity-continuation strategy that shares one ambiguity state over short arcs with verified continuity and softly links successive arcs through random-walk factors. On an approximately 20-km route from the public TEX-CUP dataset, including about 50\% deep-urban driving, the latent-pivot configuration achieves 100\% availability and a 1.6-m maximum horizontal error, with 96.16\% and 99.90\% of epochs below 1.0 and 1.5 m, respectively. The implementation is released as open-source software
I. INTRODUCTION
SeA-RVINS addresses urban localization errors that can persist when corrupted GNSS factors, false visual landmarks, or wrongly continued ambiguities enter a tightly coupled estimator. It combines early visual-track screening, bounded ambiguity continuation, and correlation-preserving robust GNSS estimation; its TEX-CUP evaluation reports full availability and low maximum horizontal error.
I. INTRODUCTION
- Corrupted GNSS factors, false visual landmarks, and wrongly continued ambiguities can persist through states and marginalization, so unreliable inputs must be rejected or downweighted.Urban GNSS is affected by blockage, multipath, and NLOS reception, while visual tracks face moving objects, repeated structures, weak texture, and poor stereo geometry.Source: §I · p. 1
- The system screens semantically unreliable and geometrically inconsistent stereo tracks before persistent landmark factors enter the graph.Its semantic-aware learned stereo frontend is one of the stated contributions.Source: §I · p. 1
A. RTK-VINS Estimation and Ambiguity Management
- The hybrid ambiguity strategy shares states over bounded, continuity-verified arcs, softly links successive arcs, and initializes independent states when continuity is broken or unverifiable.Prior RTK-VINS work established temporal ambiguity reuse; SeA-RVINS addresses partially observable continuity with bounded sharing and soft links.Source: §I · p. 1, A. RTK-VINS Estimation and Ambiguity… · p. 2
I. INTRODUCTION
- DD measurements share pivot-induced correlations, and the configurable DCS formulation preserves this structure across robust GNSS estimation.The contribution is described as a correlation-preserving robust formulation.Source: §I · p. 1
- On the approximately 20-km TEX-CUP route, SeA-RVINS achieved 100% solution availability and a 1.6-m maximum horizontal error.The route included about 50% deep-urban driving and was compared with publicly available SOTA sources.Source: §I · p. 2
B. Reliable Visual Frontends in Urban Environments
SeA-RVINS pairs a learned, semantic-aware visual factor-admission pipeline with robust double-differenced GNSS factors in a tightly coupled factor graph. The graph combines sensor constraints and active variables within a fixed-lag estimation framework.
B. Reliable Visual Frontends in Urban Environments
- SeA-RVINS combines semantic screening, learned stereo and temporal matching, and track- and frame-level motion checks before visual factors enter the graph.The cited prior approaches address semantic or geometric filtering, while this pipeline also combines learned feature extraction with learned stereo and temporal matching to reduce ambiguous static associations.Source: B. Reliable Visual Frontends in Urban… · p. 2
C. Correlation-Preserving Robust GNSS Factors
- For double-differenced GNSS factors, batch weighting uses one robust loss on the full-covariance Mahalanobis norm, while componentwise and candidate-wise weighting assign separate weights at different residual levels.The shared pivot makes candidate errors correlated; the passage identifies candidate-wise robustification that preserves this structure as the gap addressed.Source: C. Correlation-Preserving Robust GNSS… · p. 2
- SeA-RVINS compares batch, scalar, and latent-pivot configurations with DCS for DD code and phase; all recover the same full-covariance Gaussian model under unit robust weights.The scalar mode reweights full-covariance-whitened components, while the latent-pivot mode enables candidate-wise weighting.Source: C. Correlation-Preserving Robust GNSS… · p. 2, B. Reliable Visual Frontends in Urban… · p. 2
A. System Overview
- The factor graph tightly couples GNSS, IMU, and stereo observations, jointly estimating navigation states, landmarks, float ambiguities, and optional pivot-error and stationary-anchor variables.IMU preintegration links navigation states; GNSS processing forms DD code and phase factors, and Doppler provides velocity constraints.Source: A. System Overview · p. 2
- The fixed-lag objective sums measurement-factor costs and retains information from variables leaving the window in a marginalized prior.The stated cost includes prior, IMU-preintegration, bias random-walk, stereo-reprojection, DD code/phase, Doppler, ambiguity, and stationary terms.Source: A. System Overview · p. 3
B. IMU Preintegration
IMU preintegration summarizes high-rate measurements between graph epochs as a relative-motion constraint. Its residual and propagated covariance contribute to the estimator alongside bias random-walk terms.
B. IMU Preintegration
- Between lower-rate graph states, the IMU's high-rate angular-velocity and specific-force samples are summarized as one relative-motion constraint.The paper uses the standard on-manifold preintegration formulation.Source: B. IMU Preintegration · p. 3
- The preintegrated rotation, velocity, and position increments, expressed in the starting body frame, and their propagated covariance define the summarized motion measurement.These are the quantities identified for consecutive epochs in the preintegration description.Source: B. IMU Preintegration · p. 3
- The bias-corrected preintegration residual and bias random-walk terms form the IMU-related costs in the estimator.The residual follows the cited preintegration equations, and the bias random-walk covariance is specified for consecutive graph epochs.Source: B. IMU Preintegration · p. 3
C. Semantic-Aware Stereo Vision Frontend
The stereo frontend screens semantic regions and verifies stereo and temporal consistency before tracks become persistent landmark factors. Its goal is to admit semantically screened, geometrically consistent observations rather than maximize match count.
C. Semantic-Aware Stereo Vision Frontend
- The frontend prioritizes semantically screened, geometrically consistent stereo observations over maximizing the number of matches.The vision branch runs separately and is expanded in the workflow figure.Source: C. Semantic-Aware Stereo Vision Frontend · pp. 3–4, Figure 2 · p. 4
- SegFormer masks unreliable classes in both rectified views, and SuperPoint keypoints are retained only when they satisfy the corresponding image mask.The listed classes include sky, vegetation, people, and vehicles; mask erosion and buffers add conservative object-boundary margins.Source: C. Semantic-Aware Stereo Vision Frontend · p. 4
- LightGlue supplies stereo correspondences and temporal tracks, while stereo matches undergo score, epipolar, disparity, and triangulation checks.Invalid-geometry or excessive-range triangulated points are removed.Source: C. Semantic-Aware Stereo Vision Frontend · p. 4
- PnP/RANSAC checks temporal associations, and disagreement between PnP and IMU-predicted motion controls visual-factor admission.Rejected associations break tracks, which must mature again before landmark initialization; PnP support also scales stereo-factor covariance.Source: C. Semantic-Aware Stereo Vision Frontend · p. 4
- An admitted stereo observation contributes a reprojection residual, and multiple admitted observations can share one persistent landmark variable.The stereo factors use a Huber loss, with a threshold and quadratic branch specified in the paper.Source: C. Semantic-Aware Stereo Vision Frontend · p. 4
- When PnP-derived motion indicates a stop, the backend can add a zero-velocity constraint and, under weak GNSS support, tie stopped poses to a shared position anchor.The anchor has a covariance-derived prior from a latched stopped-position estimate.Source: C. Semantic-Aware Stereo Vision Frontend · p. 4
D. GNSS Measurements
The GNSS models form double-differenced measurements to mitigate common-mode errors and eliminate receiver clock and hardware biases. Because candidate measurements share a pivot, their residuals are correlated.
D. GNSS Measurements
- Double differencing combines rover–base and candidate–pivot differences to mitigate common-mode errors and remove common receiver clock and hardware biases.The pivot is selected using high elevation and C/N0 with lower multipath risk.Source: D. GNSS Measurements · p. 4
- The DD code and phase models are defined for a candidate satellite relative to a pivot satellite on each frequency.The associated model includes pivot-relative integer ambiguity and differenced multipath and residual errors.Source: D. GNSS Measurements · p. 4
- Between-satellite Doppler differencing removes receiver clock drift after Doppler observations are converted to range rate and compensated for satellite velocity and clock drift.The corrected model uses the line-of-sight unit vector.Source: D. GNSS Measurements · p. 4
- Shared-pivot residuals have correlated errors, so their covariance must represent the pivot contribution.The text describes a batch of candidate–pivot residuals sharing one pivot observation.Source: D. GNSS Measurements · pp. 4–5
E. Correlation-Preserving Robust Estimation
SeA-RVINS offers batch, scalar, and latent-pivot robust formulations for double-differenced GNSS residuals. Each retains shared-pivot dependence while assigning robust weights at a different level.
E. Correlation-Preserving Robust Estimation
- The batch formulation applies one robust weight to the full-covariance-whitened residual vector, so one outlier can downweight otherwise clean measurements in the batch.It uses the batch Mahalanobis norm.Source: E. Correlation-Preserving Robust… · p. 5
- The scalar formulation assigns separate robust weights to whitened components without adding graph variables, but dense whitening can mix candidate residuals.An outlier can therefore affect component weights.Source: E. Correlation-Preserving Robust… · p. 5
- The latent-pivot formulation adds an epoch-local Gaussian pivot variable per batch, yielding standardized scalar residuals that are conditionally independent and tied to original candidates.Marginalizing the pivot variable recovers the shared-pivot covariance.Source: E. Correlation-Preserving Robust… · p. 5
- DCS uses the batch norm, whitened component, or candidate residual as its robust argument for the respective code and phase configurations.Doppler uses Huber weighting under the selected configuration, and weights are recomputed at each relinearization.Source: E. Correlation-Preserving Robust… · p. 5
- All three formulations retain nominal shared-pivot dependence but apply robust weights to a batch norm, whitened components, or conditional candidate residuals.With fixed model variances and unit robust weights, they recover the same full-covariance Gaussian cost.Source: E. Correlation-Preserving Robust… · p. 5
F. Hybrid Ambiguity Continuation and IAR
The ambiguity strategy combines bounded constant-state sharing with soft random-walk links between successive arcs. Integer ambiguity fixing is applied only after candidate and pose validation.
F. Hybrid Ambiguity Continuation and IAR
- Verified continuity shares one ambiguity state over arcs of up to H = 3 epochs, then softly links successive states; broken or unverifiable continuity starts an independent state.This preserves short-term coupling while limiting exact state sharing.Source: F. Hybrid Ambiguity Continuation and IAR · p. 5
- A detected cycle slip, pivot change, or single-frequency measurement prevents inheriting the preceding epoch's DD ambiguity state.Cycle slips are detected in preprocessing with a dual-frequency geometry-free test.Source: F. Hybrid Ambiguity Continuation and IAR · pp. 5–6
- MLAMBDA decorrelates the float ambiguities and solves the integer least-squares problem before candidate validation.The float ambiguity estimates and their joint covariance are collected from the current FGO solution.Source: F. Hybrid Ambiguity Continuation and IAR · p. 6
- An integer candidate is accepted only when PB > 0.86, the float pose covariance passes its gate, and the position correction meets covariance and innovation bounds.If validation fails, the float estimate is retained without integer priors from that hypothesis.Source: F. Hybrid Ambiguity Continuation and IAR · p. 6
- Accepted fixed integers enter the factor graph through tight Gaussian ambiguity priors.The ambiguity random-walk and fixing factors contribute to the ambiguity cost.Source: F. Hybrid Ambiguity Continuation and IAR · p. 6
A. Dataset and Evaluation Protocol
The evaluation uses a roughly 20-km TEX-CUP route with prolonged GNSS-degraded urban driving and compares SeA-RVINS configurations with public baselines. Errors and availability are computed over the evaluation interval under stated valid-solution rules.
A. Dataset and Evaluation Protocol
- The evaluated TEX-CUP route covers approximately 20 km and 4040 GNSS epochs, with about 50% deep-urban, 40% light-urban, and 10% open-sky driving.The initial seven and final nine parked open-sky minutes are excluded.Source: A. Dataset and Evaluation Protocol · p. 6
- The sensor setup uses 1-Hz multi-constellation rover and base GNSS, a 100-Hz IMU, synchronized 10-Hz stereo images, and a 6-s fixed-lag FGO.The evaluation applies cDCS = 1.5 and a Huber threshold of 1.345.Source: A. Dataset and Evaluation Protocol · p. 6
- Table I compares eight public baselines with batch, scalar, and latent SeA-RVINS configurations, while accounting for differences in supported measurement configurations.GVINS and InGVIO use rover-only observations and serve as system-level references rather than controlled RTK-visual-inertial comparisons.Source: A. Dataset and Evaluation Protocol · p. 6
- Availability is the fraction of epochs with a valid solution, and cumulative error percentages count missing solutions as unsuccessful.RMSE, maximum error, and P95 use valid-solution epochs; Fig. 3 shows cumulative horizontal-error distributions with 1.0-m and 1.5-m thresholds.Source: A. Dataset and Evaluation Protocol · p. 6, Figure 3 · p. 6
B. Results and Discussion
All three SeA-RVINS configurations outperform the evaluated public baselines while maintaining 100% availability, with different trade-offs in average, worst-case, and used-fix accuracy. The selected downtown trajectories remain close to ground truth, consistent with the intended role of reliable, screened visual landmarks.
B. Results and Discussion
- 100% availability and 0.38–0.39 m horizontal RMSE accompany substantial gains over evaluated public baselines across all three SeA-RVINS configurations.Their horizontal errors span 1.60–2.91 m at the maximum, with 96.16–97.15% of all epochs below 1 m.Source: B. Results and Discussion · pp. 6–7, Table I · p. 7
- Batch has the lowest horizontal RMSE (0.38 m), scalar the lowest horizontal P95 (0.86 m), and latent the lowest maximum horizontal and 3-D errors (1.60 and 4.04 m).The configurations therefore trade off complexity and robustness rather than uniformly improving average accuracy.Source: Table II · p. 7, B. Results and Discussion · p. 7
- Scalar and latent raise used-fix rates from batch’s 34.23% to 47.40% and 45.59%, respectively; latent has 12 used-fix epochs with horizontal errors ≥0.5 m, versus 34 for batch and 115 for scalar.Latent also has the lowest used-fix RMSE and maximum error, 0.15 and 0.55 m.Source: Table II · p. 7, B. Results and Discussion · p. 7
- In four selected downtown Austin segments, SeA-RVINS trajectories remain close to ground truth while several baseline trajectories depart substantially, including across building footprints.These spatial examples illustrate the large-error events reported in Table I.Source: Figure 4 · p. 7, B. Results and Discussion · p. 7
- Reliable landmarks provide relative constraints under degraded GNSS, while semantic and geometric screening reduce the risk of corrupted visual factors entering the estimator.The results are consistent with the intended role of the learned frontend.Source: B. Results and Discussion · p. 7
V. CONCLUSION AND FUTURE WORK
SeA-RVINS combines a semantic-aware learned stereo frontend, hybrid ambiguity continuation, and correlation-preserving robust GNSS estimation. On the approximately 20-km TEX-CUP route, its three robust configurations achieve 100% availability, while latent-pivot has the lowest maximum horizontal error; future work includes partial IAR, robustness validation, and lane-map aiding.
V. CONCLUSION AND FUTURE WORK
- SeA-RVINS combines a semantic-aware learned stereo frontend, hybrid ambiguity continuation, and correlation-preserving robust GNSS estimation to limit unreliable measurements’ admission, influence, and temporal propagation.Source: §V · p. 7
- On the approximately 20-km TEX-CUP route, all three robust configurations achieve 100% availability and 0.38–0.39 m horizontal RMSE; latent-pivot limits maximum horizontal error to 1.60 m.Latent-pivot also achieves the lowest maximum 3-D error and the most accurate used-fix solutions.Source: §V · p. 8
- Future work includes partial IAR when the full ambiguity vector cannot be reliably fixed, dedicated robustness validation, and guarded lane-map aiding.The proposed lane-map constraints use lane centerlines and boundaries for lateral and heading information.Source: §V · p. 8