Forward-Facing Near-Infrared Adds Little to Colour for Farm-Machinery Traversability: A Site-Disjoint Evaluation of Sensor-Dependent Spatial Leakage

1. Introduction

• Sequence-level splits can place neighbouring or near-duplicate views in both training and test sets, inflating measured segmentation accuracy and rewarding site memorisation.The cited benchmarks did not separate training and evaluation by location.Source: §1 · pp. 1–2
• The sequence-level split leaves frames from the same plots on both sides, making the apparent NIR paddy-boundary advantage indistinguishable from location overlap.The corpus pairs colour and NIR during rice-paddy and dry-field driving, while separate sequences revisit plots.Source: §1 · p. 2
• The study tests colour, NIR, a capacity-matched greyscale luminance control, and colour–NIR fusion under sequence-level and disjoint-super-site protocols.The luminance control isolates a spectral-band gain from the added single intensity map; scores use colour annotations, NIR annotations, and their pixel intersection.Source: §1 · p. 2
• Under site-disjoint evaluation, NIR beats colour on none of five traversability classes, and fusion provides no consistent gain; two single-site leads reverse under site rotation.The authors conclude that NIR remains unproven for daytime paddy and dry-field driving.Source: §1 · p. 2

2. Related work

• Blocked and leave-one-location-out validation address optimistic accuracy from spatial autocorrelation, including in convolutional segmentation networks.The paper applies this audit-and-correction approach to forward-facing agricultural traversability.Source: §2 · p. 2
• Spatial leakage can distort which predictors appear important, and this study examines the analogous question of apparent advantage between whole sensor modalities.The study measures transfer to an unseen site, not map accuracy over a sampled region.Source: §2 · pp. 2–3
• Earlier road and outdoor segmentation studies report NIR gains, but do not establish that the band beats a same-capacity brightness input.The luminance control holds the single-map input structure fixed while varying the spectral band.Source: §2 · p. 3
• Existing paired-band datasets do not supply the same evaluation setting: GOOSE uses a split balanced by seasons, weather, and classes rather than locations, while agricultural paired imagery is aerial or downward-facing.The study uses a forward-facing agricultural corpus and constructs a leakage-aware split.Source: §2 · p. 3

3.1. Dataset, annotations, and site recovery

• The corpus supplies paired forward-facing colour and NIR imagery with separate pixel annotations; the study uses jointly annotated frames and excludes orchard recordings lacking the target ground classes.The five traversability classes cover paddy and field ground; road and trees serve as controls, while standing rice is reported only for completeness.Source: §3.1 · p. 3
• GPS-track clustering recovers three super-sites, and the sequence-level split is compared with site-disjoint protocols because separate sequences revisit the same plots.Sequences join a super-site when their GPS tracks are within 11 m; site-disjoint partitions keep locations separate.Source: §3.1 · p. 3, §3.3 · p. 4

3.2. Input configurations and models

• Colour, NIR, and Rec. 601 luminance are matched single-network inputs, while fusion uses two streams; the luminance control isolates spectral-band effects from single-map input structure.The NIR-versus-control comparison varies the band while holding input structure fixed; fusion has at least 1.8 times the parameters of the matched configurations.Source: §3.2 · p. 4
• The study tests whether findings depend on model scale or architecture by repeating runs with larger SegFormer and UPerNet/ConvNeXt-V2-T configurations.The primary encoder is SegFormer MiT-B0; the additional models vary encoder size and architecture family.Source: §3.2 · p. 4

3.3. Evaluation protocols and statistical criteria

• The leave-one-site-out rotation holds out each super-site, while a paddy-restricted rotation and matched sequence-level comparator separate site blocking from environment shift.Single-environment training and scoring on both familiar and unfamiliar environments further distinguish environment change from site change.Source: §3.3 · p. 4
• Leakage-controlled comparisons use the pixel intersection of the two annotations, and a configuration beats another only if corrected paired tests, a zero-excluding confidence interval, and a single-annotation check agree.The intersection includes pixels only where colour and NIR annotations agree, preventing annotation-specific matches from inflating a comparison.Source: §3.3 · p. 5
• Leakage inflation is the paired per-frame IoU difference between models trained with shared locations and under site-disjoint training, scored on frames both protocols evaluate.Pairing on common frames separates leakage from site difficulty; the whole driving sequence is the bootstrap resampling unit because adjacent frames are autocorrelated.Source: §3.3 · p. 5
• The study reports descriptive per-class accuracies and uses sequence-clustered intervals; four-sequence paired contrasts yield only 35 distinct resampled means, limiting endpoint resolution.Some criteria are treated as directional rather than results when resampled mass lies within three Monte Carlo standard errors of the interval tail.Source: §3.3 · pp. 5–6

4.1. No configuration beats colour once the sites are held apart

• Under the site-blocked split, near-infrared beats colour on none of the five target classes, and fusion yields no consistent gain.Fusion exceeds colour on the tilled paddy surface by +0.033 (95% CI [+0.017, +0.048]), while colour exceeds fusion on the field furrow by +0.076 (95% CI [+0.014, +0.107]); the other three classes are comparable.Source: §4.1 · p. 6
• Near-infrared exceeds the luminance control on the tilled paddy surface by +0.030, but the interval spans zero; on the untilled surface, +0.020 has an interval of [+0.000, +0.048].Across held-out sites, the untilled-surface contrast reverses to −0.038 on one fold and is indistinguishable from zero at the site level (p = 0.92).Source: §4.1 · p. 7
• Near-infrared’s pooled gain over luminance meets the protocol criterion, but fold margins change sign and the site-level test gives p = 0.33.Rotating the held-out site also makes the fusion-over-colour gain change sign: margins are +0.032, +0.010, and −0.075, with site-level p = 0.77.Source: §4.1 · p. 7

4.2. The apparent paddy-edge advantage is site leakage concentrated in near-infrared

• The sequence-level split puts every super-site in both partitions, and its leakage inflation is positive on almost every cell, with near-infrared largest on all but one shown class.The split therefore evaluates models on sites represented in training; holding sites apart lowers scores across multiple classes and configurations.Source: §4.2 · p. 7
• Under the sequence-level split, paddy-edge IoU is 0.339 for near-infrared versus 0.009 for colour, but near-infrared scores 0.015 under the site-blocked split.On the 322 frames scored by both protocols, near-infrared’s paired inflation is +0.067; its paired margin over colour changes from +0.127 under the sequence split to −0.013 (95% CI [−0.028, −0.000]) under site blocking.Source: §4.2 · p. 7
• On identical held-out-site frames, the seen-site minus unseen-site score difference is +0.067 for near-infrared, +0.027 for luminance, and −0.008 for colour.The figure reports pooled intersection-IoU on intersection ground truth; the authors attribute the gap to site recognition rather than segmentation difficulty.Source: §4.2 · p. 8, Figure 1 · p. 8
• The apparent near-infrared ordering is not confirmed under the other ground truth: only the intersection ground truth resolves any step in the configuration ordering.The own-annotation gaps are +0.139 for luminance, +0.119 for near-infrared, and +0.059 for colour, but these are not all paired comparisons.Source: §4.2 · p. 8
• Tests of brightness, annotation, and camera alignment do not explain the near-infrared leakage concentration; on near-infrared’s own annotation, paired paddy-edge inflation is +0.119.The reported checks find brightness and the supervising annotation no better than baseline at identifying the site; removing measured camera offset does not restore agreement on the evaluation frames.Source: §4.2 · p. 9

5.1. The answer for a forward-facing sensor suite

• Near-infrared beats colour on none of the five traversability classes under site-disjoint evaluation, and neither fusion nor the second camera’s single-site leads show a gain that survives site rotation.For daytime paddy and dry-field driving, the second camera remains unproven for this task.Source: §5.1 · p. 9

5.2. What a leaking split decides, and what a benchmark must do about it

• Sequence-level splitting still inflated near-infrared’s paddy-edge accuracy, so separating recordings without blocking sites is insufficient for sensor comparisons.A benchmark that omits site blocking risks unsafe procurement or design decisions.Source: §5.2 · p. 9
• Near-infrared memorised sites more than colour and the luminance control, concentrating leakage in the spectral band rather than the added intensity map.The stated attribution rests on the intersection ground truth.Source: §5.2 · p. 9
• Pooling per-frame differences across folds can conceal site-specific reversals because frame-count weighting lets one large site carry a pooled pass.The two leads that reverse under held-out-site rotation are the evidence against treating the pooled result as general.Source: §5.2 · p. 9
• The recommended safeguards are held-out-site rotation, a capacity-matched luminance control, sequence-resampled confidence intervals, and ground truth formed from annotation intersection.On the same data, the frame-level interval for near-infrared’s margin over the control excludes zero, while the sequence-level interval spans it.Source: §5.2 · p. 9

5.3. Boundaries

• The leakage attribution is conditional because all encoders use visible-band pretraining, which may make near-infrared rely more on site cues; near-infrared pretraining could separate that reliance from a band effect.The comparison tests near-infrared under a 30-epoch budget, not the full potential of the band.Source: §5.3 · p. 10
• The evidence comes from one daylight, clear-weather corpus in one Korean province; balanced site-disjoint rotation was possible for rice paddy, not dry-field driving.The larger fusion effect is on a dry-field class without rotation, and the cross-camera offset makes thin-class accuracies a floor.Source: §5.3 · p. 10

• No input configuration beats colour consistently on any traversability target class in the site-disjoint evaluation.The four configurations are colour, near-infrared, luminance control, and their fusion.Source: §6 · p. 10
• Both of the second camera’s leads at one site change sign when the held-out site is rotated.The leads were fusion over colour and near-infrared over the luminance control.Source: §6 · p. 10
• Under the corpus’s sequence-level split, near-infrared’s advantage reflects site recognition, and the second camera remains unproven for daytime paddy and dry-field driving.Near-infrared gains more from having seen a site than colour or the capacity-matched luminance control does.Source: §6 · p. 10

• Table A.1 reports accuracies for every annotated class scored under both the site-blocked and sequence-level protocols.The sequence-level accuracies are shown separately because the protocols score different frames, so subtracting their blocks does not recover the paired leakage inflation.Source: Appendix A · p. 10
• Two classes appear only in Table A.1: in-plot trees and standing rice.In-plot trees have too few site-blocked validation frames to score; standing rice is neither a traversability target nor a control.Source: Appendix A · p. 10
• Table 1 pairs site-blocked accuracy with leakage inflation, whereas Table A.1 carries the protocol-specific accuracies.The separation avoids implying that subtracting scores over different frames yields the paired quantity.Source: Appendix A · p. 10

• Table B.1 reports colour and near-infrared accuracies after training on one driving environment and scoring on both held-out-super-site environments.It includes each class with enough held-out frames in an environment to score.Source: Appendix B · p. 10
• Table A.1 reports per-class intersection-over-union under the site-blocked and sequence-level protocols as five-seed means with standard deviations.The site-blocked block repeats Table 1; the sequence-level block supplies the accuracies used to measure leakage inflation.Source: Table A.1 · p. 11
• In-plot trees are not evaluable under the site-blocked split, which retains only nine validation frames; standing rice is included for completeness, not as a target or control.Source: Table A.1 · p. 11

Appendix C. Encoder size and architecture family

• Table C.1 covers both Section 4 contrasts across every input configuration and model.The contrasts are paddy-edge leakage inflation and NIR’s accuracy margin over the capacity-matched luminance control on tilled paddy.Source: Appendix C · p. 11
• The paddy-edge contrast measures leakage inflation across the configurations and models.The table carries every cell for this contrast.Source: Appendix C · p. 11
• The tilled-paddy contrast measures NIR’s accuracy margin over the capacity-matched luminance control.The NIR cell is where the protocol criteria are declared.Source: Appendix C · p. 11

Appendix D. What does not explain the concentration

• Table D.1 measures whether frame brightness alone identifies the super-site.It also tests whether the supervised annotation identifies the super-site and how camera offset erodes paddy-edge intersection on the scored frames.Source: Appendix D · p. 11
• The table tests whether the configuration’s supervisory annotation identifies its super-site.This is one of the three mechanism measurements reported in the section.Source: Appendix D · p. 11
• The table measures how cross-camera offset erodes paddy-edge intersection on the frame sets used for comparisons.The section presents these measurements as mechanisms that could explain leakage concentration without appealing to the spectral band, not as claimed results.Source: Appendix D · p. 11