Source-linked AI summary
Band-Selection Stability and Semantic Segmentation Performance: A Study on Hyperspectral City
Jiarong Li, Imad Ali Shah, Enda Ward, Martin Glavin, Edward Jones, Brian Deegan
TL;DR
High-dimensional hyperspectral imaging motivates band selection, but the relationship between selection stability and segmentation performance is underexplored. The study evaluates six methods across repeated class-balanced ROIs and SSMs; stability does not reliably predict performance, despite competitive selected-band results.
Problem
Band selection is motivated by HSI’s high dimensionality, while the relationship between stability under repeated class-balanced ROI sampling and downstream SSM performance remains underexplored.
Source: §1 · p. 1, §2.3 · p. 2Method
Six band-selection methods produced 60 top-25 subsets across ten class-balanced ROIs, with top-K subsets evaluated against 128-band baselines using three SSMs.
Source: §1 · p. 1Results
Stability did not reliably predict segmentation performance: Sim-LP was most stable, while Sim-LP and JMIM+CSNR achieved the six SSM-metric optima, with gains up to 2.01 mIoU and 1.72 mF1 points over 128-band baselines.
Source: §5 · p. 5Takeaways & Limitations
Stability characterizes reproducibility but is not a reliable performance indicator in these experiments; methods should be assessed across repeated samples, subset sizes, and downstream SSMs.
Source: §5 · p. 5Conclusions are limited to one urban HSI dataset and pairwise Jaccard similarity; generalization across datasets and alternative stability measures remains future work.
Source: §5 · p. 5
Abstract
from arXiv · showhide
Resource constraints make high-dimensional hyperspectral imaging challenging in autonomous perception, motivating the use of band selection methods. However, the sensitivity of band-selection methods to sampled data and their relationship to semantic segmentation models (SSMs) remain underexplored. This study evaluates six band selection methods on ten independently sampled, class-balanced region-of-interest (ROI) sets, yielding 60 top-25 band subsets from the Hyperspectral City V2 (128 bands: 450-950nm) dataset. Top-$K$ bands ($K\in\{3,5, ... 13\}$) from the first three ROI sets are evaluated with three SSMs against the corresponding 128-band baseline. Experiments show that intra-method stability is method-dependent: Sim-LP shows the highest stability (pairwise Jaccard similarity) and, together with JMIM+CSNR, yields the best segmentation results. Top-$K$ based SSMs remain competitive with baselines, with gains of up to 2.01 mIoU and 1.72 mF1 points, and 18-22x faster CPU inference for $K=9$. However, performance does not improve monotonically with $K$, and stability shows no consistent association with SSM performance. These findings suggest that intra-method stability is informative but an unreliable indicator of downstream segmentation performance, highlighting the need to evaluate band-selection methods across repeated samples, subset sizes, and SSMs.
1. INTRODUCTION
Hyperspectral imaging offers material information beyond RGB for urban semantic segmentation, but its high dimensionality, computational demands, and spectral redundancy motivate selecting fewer bands. The study tests whether selection methods vary across class-balanced ROI samples and whether greater stability corresponds to better segmentation.
1. INTRODUCTION
- HSI can provide material information beyond conventional RGB in urban-driving scenes, but its high dimensionality, computational cost, and spectral redundancy make band selection useful.Band selection aims to retain a small, informative set of wavelengths.Source: §1 · p. 1
- The study asks how common band-selection methods vary across repeated class-balanced ROI samples and whether intra-method stability corresponds to SSM performance.The motivation includes severe H-City class imbalance: Road accounts for 22.43% of labeled pixels, versus 0.02% for Train.Source: §1 · p. 1
- Six methods are applied to ten class-balanced ROIs to produce 60 top-25 subsets, then evaluated across subset sizes and three SSMs against 128-band baselines.The study also examines stability–performance relationships and accuracy–inference-time trade-offs, with frequency consensus as an exploratory aggregate baseline.Source: §1 · p. 1
2. RELATED WORK
Prior work applies HSI and band selection to urban and vulnerable-road-user perception, while selection methods differ in supervision and redundancy handling. The relationship between stability under repeated class-balanced ROI sampling and downstream SSM performance remains underexplored.
2.1. HSI and Band-Selection in Autonomous Driving
- Prior work investigated HSI for urban-scene understanding and annotation refinement, while related studies examined automotive HSI segmentation and vulnerable-road-user perception.Embedded selectors learn bands jointly with downstream models and can depend on architecture, initialization, and optimization, unlike the sampling-stability focus here.Source: §2.1 · pp. 1–2
2.2. Hyperspectral Band-Selection
- mRMR balances class relevance and redundancy, CMIM and JMIM use mutual-information criteria, and Sim-LP favors bands poorly reconstructed from those already selected.JMIM+CSNR combines JMIM with patch-based contrast signal-to-noise ratio; JMIM+CSNR+Corr adds Pearson-correlation duplicate removal.Source: §2.2 · p. 2
2.3. Band-Selection Stability vs SSM Performance
- Although prior work examined band-selection stability and classification accuracy, its relationship to downstream SSM performance under repeated class-balanced ROI sampling remains underexplored, especially in automotive HSI.Pairwise Jaccard similarity provides a direct overlap measure for fixed-size subsets.Source: §2.3 · p. 2
3. EXPERIMENTAL PROTOCOL
The protocol samples class-balanced ROIs from H-City, measures repeated-selection stability, constructs a frequency-consensus ranking, and evaluates selected bands with SSMs and stability–performance comparisons.
3.1. Dataset and Class-Balanced ROI Sampling
- Ten sampling seeds generated class-balanced ROIs from H-City’s 128 bands and 19 annotated urban classes, using training data for ROI sampling and SSM training and validation data for evaluation.The original validation and test sets had different annotation granularity.Source: §3.1 · p. 2
3.2. Intra-Method Stability Characterization
- Six methods produced top-25 subsets for each ROI; stability was summarized by the median and interquartile range of 45 pairwise Jaccard similarities per method.The random-subset reference for two 25-of-128 subsets is approximately 0.11.Source: §3.2 · p. 2
3.3. Frequency-Based Consensus Construction
- Frequency consensus scores bands by their recurrence across the four retained methods, ranks bands by score, and forms subsets from the first K bands.The consensus uses JMIM, JMIM+CSNR, JMIM+CSNR+Corr, and Sim-LP; its subsets are used for SSM training and evaluation.Source: §3.3 · p. 2
3.4. Semantic-Segmentation Validation
- Rankings from three ROI sets were truncated at K bands and evaluated with U-Net, DeepLabV3+, and SegFormer, using three training seeds per subset and the corresponding 128-band baseline.Individual-method results aggregate nine evaluations; consensus and baseline results use three training seeds, and performance is measured by mIoU and mF1.Source: §3.4 · pp. 2–3
- Figure 1 depicts method-specific top-25 band recurrence across ten class-balanced ROIs, with color indicating selection count.Source: Figure 1 · p. 3
3.5. Method-Level Stability: Performance Comparison
- Method-level selected-minus-baseline mIoU and mF1 differences were averaged across three SSMs and six subset sizes, then compared with mean pairwise Jaccard stability.Frequency consensus was excluded because its single aggregate ranking has no intra-method stability value; Figure 2 presents pairwise Jaccard similarities and a 0.11 random-subset reference.Source: §3.5 · p. 3, Figure 2 · p. 3
4. RESULTS AND DISCUSSION
Across the evaluated subsets, Sim-LP and JMIM+CSNR lead segmentation results, and K = 9 offers a favorable aggregate speed–accuracy trade-off. Selection stability varies by method but has no monotonic relationship with baseline-relative segmentation performance.
4.1. Intra-Method Band-Selection Stability
- Band recurrence patterns differ across methods: Sim-LP selections are sparse across visible and NIR ranges, while mRMR recurrence clusters near both spectral boundaries and JMIM+CSNR+Corr in blue-green.CMIM's recurrent selections span much of the spectrum, with several of its highest selection counts in the NIR.Source: §4.1 · p. 3
- Sim-LP has the highest median pairwise Jaccard similarity and a relatively narrow interquartile range; CMIM has the lowest median, with its lowest values near the 0.11 random-subset reference.JMIM, JMIM+CSNR, and JMIM+CSNR+Corr form a relatively high-agreement group below Sim-LP, while mRMR falls between that group and CMIM.Source: §4.1 · p. 3
4.2. Segmentation Performance of Band Subsets
- K = 9 yields three of six best results and 25 of 36 above-baseline results, but performance is non-monotonic and no subset size is optimal across all SSMs and metrics.The frequency consensus exceeds baseline in only one of 36 results and never surpasses the best individual method.Source: §4.2 · pp. 3–4
- Sim-LP and JMIM+CSNR achieve all six SSM-metric best results; SegFormer gains peak at 2.01 mIoU points (JMIM+CSNR, K = 7) and 1.72 mF1 points (Sim-LP, K = 5).Table 1 reports mean ± SD results by SSM, method, and subset size, with maxima marked in bold.Source: §4.2 · p. 3, Table 1 · p. 4
4.3. Inference-Time and Accuracy Trade-off
- 18.7–22.0× faster CPU input-layer inference than 128 bands is achieved at K = 9 across the three SSMs, with averaged mIoU and mF1 differences within roughly ±0.1 percentage points.This aggregate masks method-level variation: five methods remain within about 1 point of baseline, while JMIM+CSNR+Corr trails by 2.44–2.74 mIoU and 1.72–2.21 mF1 points.Source: Figure 3 · p. 4, §4.3 · p. 4
4.4. Method-Level Stability–Performance Association
- No monotonic relationship links intra-method stability with baseline-relative segmentation performance; high-stability JMIM+CSNR+Corr has the largest deficits, while low-stability CMIM is positive on both metrics.Sim-LP combines the highest stability with method-level differences of approximately 0.34 mIoU and 0.55 mF1 points.Source: Figure 4 · p. 4, §4.4 · p. 4
5. CONCLUSION
Repeated class-balanced ROI sampling shows that band-selection stability depends on the method but does not reliably predict downstream segmentation performance. Sim-LP and JMIM+CSNR produced the strongest segmentation results, while K = 9 offered a favorable efficiency–accuracy trade-off.
5. CONCLUSION
- Up to 2.01 mIoU and 1.72 mF1 points over 128-band baselines were achieved among the six SSM-metric optima by Sim-LP and JMIM+CSNR.Sim-LP was the most intra-method stable; the relatively stable JMIM+CSNR+Corr had the largest performance deficits, while the least stable CMIM had positive average baseline-relative differences in both metrics.Source: §5 · p. 5
- At K = 9, CPU input-layer inference was 18.7–22.0× faster while performance remained close to baseline, despite non-monotonic performance across subset sizes.Three of the best results occurred at K = 9; the others occurred once each at K ∈ {3, 5, 7}.Source: §5 · p. 5
- Intra-method stability characterizes reproducibility but is not a reliable indicator of downstream segmentation performance in these experiments.The frequency-consensus ranking derived from more stable methods did not outperform individual methods.Source: §5 · p. 5
- The conclusions are limited to one urban HSI dataset and pairwise Jaccard similarity; the stability–performance relationship remains to be tested on other datasets and measures.Source: §5 · p. 5