Source-linked AI summary
SIGMA-Lane: Scale-pyramId Gated MAmba for Temporally Consistent Video Lane Detection
Tiancheng Zhang, Mengmeng Wang, Yan Gao, Xiangjie Kong, Guojiang Shen, Jiaxin Du
TL;DR
Video lane detection can suffer persistent errors when occluded observations contaminate recurrent temporal state. SIGMA-Lane gates SSM writes and residual fusion, combines aligned historical retrieval, and improves temporal stability under heavy occlusion while retaining competitive detection quality on VIL-100 and OpenLane-V.
Problem
Video lane detectors need temporally consistent predictions, but prolonged vehicle occlusions can corrupt recurrent state and affect later predictions.
Method
SIGMA-Lane uses coordinate-consistent alignment, SSM-consistent dual-gating of writes and residual fusion, and Structural Spatial Retrieval for temporal filtering and lane-structure recovery.
Results
SIGMA-Lane improves temporal stability under occlusion with competitive per-frame detection quality on VIL-100 and OpenLane-V.
Takeaways & Limitations
The results indicate that directly controlling information entering and leaving the temporal update is useful for occlusion-robust video lane detection.
Takeaways & Limitations
The theoretical analysis assumes uniformly contractive conditioned dynamics and bounded write matrices.
Abstract
from arXiv · showhide
Video lane detection requires predictions that remain stable across frames, yet severe vehicle occlusions can break temporal cues. In streaming recurrent models, corrupted observations may enter the hidden state and produce errors that persist into later frames. Existing occlusion-aware refinements usually provide obstacle masks as auxiliary inputs, so the state-update path is only indirectly protected. We propose SIGMA-Lane, which treats this failure mode as state contamination in State Space Model (SSM)-based temporal modeling. SIGMA-Lane places occlusion-aware gates on the SSM write and residual-fusion paths, controlling how current observations enter temporal memory and are fused back after temporal propagation. After coordinate-consistent affine alignment, the model combines two complementary paths: SSM-consistent dual-gating for temporal filtering and Structural Spatial Retrieval (SSR) for recovering missing lane structure from aligned historical priors. Experiments on VIL-100 and OpenLane-V show improved temporal stability under heavy occlusion, with competitive F1 and mIoU scores.
1 Introduction
SIGMA-Lane addresses temporal instability in video lane detection by treating occlusion-corrupted observations as state contamination in SSM-based recurrent modeling. It combines coordinate-consistent dual-gating for temporal filtering with Structural Spatial Retrieval for recovering missing lane structure.
- Motivation: Video lane detection must maintain temporally consistent predictions, but occlusion-corrupted observations can enter recurrent memory and leave errors across subsequent frames.In the SSM update, the current observation enters through the write term, allowing corrupted features to persist in hidden state.
- Method: SIGMA-Lane gates both the SSM write path and residual-fusion path under occlusion.The dual-gating mechanism reduces the effective write amplitude at the input stage and controls residual fusion after temporal propagation.
- Method: Lane-Guided Affine Warp applies one affine transform to features, obstacle masks, and lane masks, keeping gating signals coordinate-consistent.This alignment keeps the gating signal in the same coordinate frame as the feature being modulated.
- Method: The dual-path design separates Mamba-based temporal propagation from Structural Spatial Retrieval, which recovers lane structure from aligned historical priors.The two paths address temporal-state contamination and loss of lane topology with different modules.
- Results: On VIL-100 and OpenLane-V, SIGMA-Lane achieves competitive accuracy with lower flickering and missing rates.Ablations and controlled contamination analysis attribute a large share of the improvement to dual-gating.
2 Related Work
Related work spans segmentation, parametric, and anchor-based lane representations, while video methods use either window-batched aggregation or recurrent state propagation. These approaches respectively face latency or hidden-state contamination limitations under occlusion, motivating obstacle-aware protection of Mamba’s write and residual-fusion paths.
- Lane Detection Representations: Lane detectors use segmentation-based, parametric, and anchor-based representations for complex road scenes.Segmentation methods apply pixel-wise classification; parametric methods use polynomials or Bézier curves, while anchor-based detectors use predefined line priors.
- Temporal Context in Video Lane Detection: Window-batched video detection incurs high latency without reusing past computation, whereas recurrent propagation supports streaming but lacks protection against occlusion-driven state contamination.
- Temporal Context in Video Lane Detection: Single-frame detectors perform well on static benchmarks but cannot exploit temporal context to infer lanes hidden by occlusion.
- Temporal Context in Video Lane Detection: SIGMA-Lane complements Mamba’s internal selectivity with an obstacle prior at the write and residual-fusion paths.
3 Method
SIGMA-Lane replaces OMR’s aggregation module with Mamba2-based temporal propagation and spatial retrieval, addressing state contamination and coordinate inconsistency. Its aggregation pipeline uses affine alignment, SSM-consistent dual-gating, and SSR to stabilize temporal lane features under occlusion.
- Aggregation module: SIGMA-Lane retains OMR’s encoder and decoder but replaces its ConvLSTM aggregation with Mamba2 temporal propagation and spatial retrieval.ResNet18 extracts multi-scale features, while the decoder predicts probability and coefficient maps for lane-instance extraction.
- Video memory: The streaming memory bank stores past aggregated features, obstacle masks, and the previous lane mask for alignment and guidance.A geometry-aware start token initializes the recursive process at cold start.
- Temporal filtering and retrieval: The scale-pyramid temporal design splits features into high- and low-resolution branches, applies gated Mamba2 modeling, and merges them through Add&Norm before SSR retrieves missing lane structure.SSR uses current features to retrieve spatial context from warped historical features enhanced by the previous lane mask.
- SSM-Consistent Dual-Gating: SSM-Consistent Dual-Gating controls both the SSM write path and residual-fusion path with obstacle-aware input and output gates.Input gating suppresses occluded observations before state writing, while output gating down-weights contaminated current features and preserves propagated output.
- Coordinate alignment: Affine alignment synchronizes adjacent-frame feature, lane-prior, and obstacle-mask coordinates before temporal scanning, preventing tokens at different physical locations from being mixed.The local short-range transformation is designed for streaming updates, where only the most recent transition must be synchronized.
4 Experiments
SIGMA-Lane is evaluated on VIL-100 and OpenLane-V using per-frame accuracy metrics and temporal inconsistency rates. Results show strong VIL-100 performance, improved robustness under stricter occlusion-sensitive evaluation, and benefits from gated temporal aggregation, alignment, and retrieval.
- Evaluation Protocol: SIGMA-Lane is evaluated on VIL-100 and OpenLane-V using mIoU, accuracy, F1, and temporal flickering and missing rates.Per-frame metrics are reported at IoU thresholds 0.5 and 0.8, while lower RF/RM values indicate fewer inconsistent adjacent-frame detections.
- VIL-100 Results: 0.940 F1@0.5, 0.956 accuracy, and 0.801 mIoU are the best reported SIGMA-Lane results on VIL-100.It also achieves the lowest flickering and missing rates, reducing RF/RM by 41.0%/45.5% versus LaneTCA.
- Occlusion Robustness: 0.384 RM@0.8 is reported at the stricter threshold, where SIGMA-Lane improves F1@0.8 by 0.091 over OMR.The results are attributed to streaming state propagation that explicitly filters occluded writes, preserving valid lane instances when occlusion suppresses current-frame evidence.
- Ablation Studies: 0.023/0.030 RF/RM is reached by the full model after adding alignment, retrieval, and initialization.Replacing OMR’s ConvLSTM with standard Mamba2 gives a backbone-swap reference; input gating reduces corrupted writes, while output gating reduces noisy residual fusion.
- Sensitivity and Stress Tests: 6.6× larger degradation occurs without gates when structured memory perturbations raise RM@0.5 from 0.162 to 0.182 for SIGMA-Lane and from 0.236 to 0.367 for the ungated control.Mask perturbations have limited effect, indicating that placing the obstacle prior on write and residual-fusion paths is the critical operation.
5 Conclusion
SIGMA-Lane frames persistent-occlusion video lane detection failures as SSM state contamination, where corrupted observations enter temporal memory and affect later predictions. It addresses this through gated SSM write and residual-fusion paths, coordinate-consistent alignment, SSR, and geometry-aware initialization.
- 5 Conclusion: SIGMA-Lane identifies state contamination as occlusion-corrupted observations entering temporal memory and affecting later video lane predictions.The paper studies video lane detection under persistent occlusion from the perspective of SSM state updates.
- 5 Conclusion: SIGMA-Lane gates the SSM write and residual-fusion paths to address contamination during temporal propagation.Coordinate-consistent alignment, SSR, and geometry-aware initialization further support temporal propagation and structural recovery.