Tracking the Invisible: Self-Supervised Wheel Track Detection with Thermal-RGB Fusion

I. INTRODUCTION

• Snow can obscure lane lines and road structures, while changing lanes and road features undermine map-based and vision-based navigation.The cited introduction describes localization and navigation difficulties for traditional systems, LiDAR/HD maps, vision, and LiDAR SLAM.Source: §I · p. 1, Figure 1 · p. 1
• Following existing ruts can reduce lateral slippage, vibration, and instability while improving traction and reducing energy spent compacting terrain.These benefits are described for off-road driving across loose terrain including snow.Source: §I · p. 1
• Generic traversability estimation identifies drivable regions but misses the snow-driving practice of following tracks for safer, more stable travel.The paper motivates specialized wheel-track detection instead of generic drivable-area estimation.Source: §I · p. 1
• RGB-thermal fusion targets snow-covered roads where wheel tracks are indistinct in RGB images and thermal differences distinguish compacted tracks from loose snow.The paper projects recorded wheel-track trajectories onto image frames to reduce manual labeling.Source: §I · pp. 1–2
• The method shows robust wheel-track detection on WADS under snow occlusion, low contrast, and homogeneous textures, with consistent gains over baseline approaches.The supplied introduction reports these outcomes without providing metric values.Source: §I · p. 2

A. Self-supervised Traversability Estimation

• Traversability research has shifted from basic terrain features toward deep-learning classification and self-supervision, with annotation needs limiting adaptation to unfamiliar or dynamic settings.Examples include one-class classification, autoencoders, and normalizing flows.Source: A. Self-supervised Traversability… · p. 2

B. Wheel Tracks Detection

• Wheel-track methods include LiDAR rut detection, hand-crafted image processing, elevation-map detection, and LiDAR bump-and-dip processing.The review notes customization and processing-stage complexity as constraints on some approaches.Source: B. Wheel Tracks Detection · p. 2
• Snow-track systems use manually labeled data, including a CNN approach [22] and a YOLOv5 pipeline, motivating less annotation-intensive and more adaptable methods.The review describes labor-intensive annotation and limits to generalization or output quality in these systems.Source: B. Wheel Tracks Detection · p. 2

C. IR Thermal Camera based Navigation

• Prior work uses thermal imaging for crop-row navigation, nighttime navigation, and road or pedestrian segmentation in snow and harsh weather.The cited examples combine thermal sensing with CNN segmentation, model-predictive control, or RGB-thermal fusion.Source: C. IR Thermal Camera based Navigation · p. 2

D. Research Gap

• Snow tracks can overlap, vary in depth, and become irregular through imperfect driving, new snowfall, and wind, unlike more controlled track settings.These variable characteristics make snowy wheel tracks difficult to distinguish.Source: D. Research Gap · p. 2

III. METHODS

• The proposed low-cost neural-network approach uses thermal and textural signatures in compacted snow to identify tracks in visually degenerate environments.The paper presents these signatures as guidance for autonomous systems in otherwise featureless terrain.Source: §III · pp. 2–3

A. Wheel Tracks Labeling

• Recorded GNSS trajectories are converted to wheel-terrain contact points and projected into camera coordinates as self-supervised labels; precise contact points average under 5% of pixels.Because opposite-lane tracks lack corresponding trajectory labels, the dataset adds manual labels; annotation averaged 8–10 seconds per image and about 2.5 hours for 951 images.Source: A. Wheel Tracks Labeling · p. 3

B. Input Modalities and Registration

• Figure 2 distinguishes GNSS-derived ego-path labels in green from manually annotated opposite-path tracks in red.The caption notes that omitting opposite-path tracks can degrade training performance.Source: Figure 2 · p. 3
• The four-channel RGB-I input retains RGB texture and chromatic cues alongside LWIR, supporting distinctions among compacted snow, slush, ice-water mixtures, and exposed asphalt.The framework uses the RGB and LWIR data natively provided by WADS rather than a panchromatic input.Source: B. Input Modalities and Registration · p. 3
• RGB and LWIR are downsampled to 512×512 and aligned with a static homography, assuming the driving path and road surface approximate a flat plane.The text states that parallax on vertically prominent structures is theoretically possible but negligible for the road-surface region of interest.Source: B. Input Modalities and Registration · p. 3

C. Wheel Tracks Learning

• The model uses a ResNet encoder to extract hierarchical features and contextual information from the combined four-channel RGB-I image.The encoder's deep residual layers provide multi-scale receptive fields for context aggregation.Source: C. Wheel Tracks Learning · p. 3
• Severe class imbalance between wheel-track pixels and background regions challenges model training.Source: C. Wheel Tracks Learning · p. 4
• Attention gates adaptively weight encoder features to amplify wheel-track pixels and suppress background.The gate combines encoder and decoder features to produce spatial attention weights.Source: C. Wheel Tracks Learning · p. 4
• Parallel atrous convolutions capture wheel tracks at varying scales while preserving fine details through expanded receptive fields.Their scale-specific features are concatenated into a multi-scale representation.Source: C. Wheel Tracks Learning · p. 4

D. Loss Functions

• Focal loss focuses on hard-to-classify pixels and down-weights easy pixels to address severe class imbalance.Source: D. Loss Functions · p. 4
• Boundary loss penalizes discrepancies between predicted and projected wheel-track boundaries to improve edge accuracy.The boundary term uses differences between gradients of ground-truth and predicted segmentation maps.Source: D. Loss Functions · p. 4
• The total loss combines focal and boundary losses, with w1 = 1 and w2 = 1 selected to balance their contributions.Source: D. Loss Functions · p. 4

IV. EXPERIMENTS

• Ablations report that thermal imaging improves detection in low-contrast scenarios where RGB-only methods fail, and that attention gates with atrous blocks yield significant gains.The evaluation includes white-out visibility, low-contrast snow, and variable track geometries.Source: §IV · p. 4

A. Dataset

• WADS provides winter-driving data from heavy snowfall and occasional white-outs, with LiDAR, visible and infrared cameras, GNSS, and IMUs.Its rural and semi-rural settings include varied road surfaces, weather, lighting, and terrain.Source: A. Dataset · p. 4
• Augmentation expands 951 images to 7,608 training images using flips, rotations, brightness and contrast changes, scaling, noise, and perspective changes.Source: A. Dataset · p. 4
• The processing pipeline aligns enhanced RGB and thermal images, combines them as RGB-I input, and predicts a pixel-wise wheel-track probability map.The figure depicts ResNet-18 with attention gates and atrous blocks between input preparation and prediction.Source: Figure 3 · p. 5

B. Experimental Setup

• UNet, PSPNet, DeepLabv3+, and ResNet18 serve as baselines, alongside the modified ResNet18 variant, with all models trained for 10 epochs.Training uses batch size 4, 512 x 512 images, Adam, a learning-rate schedule starting at 1e-4, and L2 regularization.Source: B. Experimental Setup · p. 5
• The modifications preserve each baseline's existing multi-scale design while adding attention gates to refine feature selection.UNet and ResNet18 receive attention gates and atrous blocks; PSPNet retains pyramid pooling, while DeepLabv3+ retains ASPP.Source: B. Experimental Setup · p. 5
• Table I presents quantitative wheel-track detection results for baseline models trained on RGB-I images.Source: Table I · p. 5

C. Evaluation Metrics

• AUROC measures threshold-independent ranking, while MaxF, AP, PRE, REC, FPR, and FNR provide complementary road-detection measures.PRE, REC, FPR, and FNR are computed at the threshold corresponding to maximum F1-score.Source: C. Evaluation Metrics · p. 5

D. Experiment Results

• The method achieves superior performance across most categories compared with existing approaches, with more confident and reliable predictions.The reported quantitative and qualitative results are presented in Tables I and II and Figure 6.Source: D. Experiment Results · p. 5
• Thermal imagery reduces FNR by 69.74% versus the RGB-only baseline while maintaining a false positive rate of 0.014.Qualitative cases include thermal predictions recovering tracks missed by RGB and avoiding RGB errors on opposite-lane tracks.Source: D. Experiment Results · p. 6
• Attention gates and atrous blocks improve AUROC by 10.3%, AP by 10.58%, MaxF by 17.35%, and REC by 30.10% on average across four models.The architecture also produces larger FPR; Figure 5 compares precision-recall curves with and without these components.Source: D. Experiment Results · p. 6, Figure 5 · p. 6
• Preprocessing takes 3.89 ms on average (257.1 Hz), accounting for less than 11% of total end-to-end latency.The pipeline includes multimodal downsampling, CLAHE, and static homography alignment.Source: D. Experiment Results · p. 6
• The attention and atrous blocks raise ResNet18 latency from 26.85 ms (37.2 FPS) to 34.73 ms (28.8 FPS), while end-to-end throughput is 25.9 FPS (38.62 ms).The paper states that this processing rate meets the 15–30 Hz real-time requirement for autonomous driving.Source: D. Experiment Results · p. 6

E. Discussion

• The fused RGB-I approach achieves the best overall performance, exceeding both RGB-only and thermal-only approaches.Attention gates and atrous blocks also yield consistent gains across evaluated baseline models.Source: E. Discussion · pp. 6–7
• When wheel-track features are absent from both RGB and thermal imagery, predictions become fragmented and low-confidence.Figure 6 illustrates this limitation in its final row.Source: Figure 6 · p. 7
• The dataset covers specific winter conditions, so results validate the framework within those constraints rather than across all snow types.The discussion names deep powder, ice crusts, and melting slush as unrepresented examples, and notes that downstream-planning effects of perception errors remain important.Source: E. Discussion · p. 7

• The self-supervised pipeline targets snow-covered wheel tracks by using winter-specific navigational cues instead of broad drivable-area predictions.Source: §V · p. 7
• 69.74% reduction in false negative rate versus RGB-only baselines, with a low false positive rate of 0.014, is reported for thermal imaging.Source: §V · p. 7
• Future work includes domain adaptation for diverse terrains, illumination, and weather conditions.Examples include slush, ice patches, mud pits, flooded roads, day/night settings, snowfall, fog, and headlight glare.Source: §V · p. 7
• Future directions include vehicle-dynamics integration, temporal modeling for tracking continuity, and digital twins for broader edge-case evaluation.Source: §V · p. 8