Source-linked AI summary
Super-Resolving Unseen Hyperspectral Sensors at Any Scale via Spatial Operators
Ji-Xuan He, Guohang Zhuang, Bo Junge, Tingyi Li, Lingchen, Miaomiao Cai, Yanan Qiao, Xiujin Liu, Junfeng Fang
TL;DR
Hyperspectral super-resolution must handle sensors with different band configurations and reconstruction at arbitrary scales, a combination that remains challenging. OmniHSR predicts band-shared spatial operators, and its reported results include zero-shot transfer to unseen sensors and gains over the strongest baseline across multiple factors.
Problem
Cross-sensor generalization and arbitrary-scale reconstruction remain challenging because HSR sensors differ in band count and existing methods can require target data, extra inference, or retraining.
Source: §1 · p. 2Method
Results
0.55 dB higher average PSNR than the strongest baseline on Pavia U and Chikusei across twelve factors from ×2 to ×48; it also outperforms directly transferred baselines on six unseen datasets.
Source: §1 · p. 2Takeaways & Limitations
The paper concludes that cross-sensor HSR depends on what the network outputs, and reports transfer from 31-band ARAD to sensors with up to 145 bands without target data.
Source: §5 · p. 9The band-shared operator prior assumes every band undergoes the same spatial degradation, so one spatial operator should serve all bands.
Source: §3.1 · p. 4
Abstract
from arXiv · showhide
Achieving cross-sensor generalization and arbitrary-scale reconstruction with a single model remains challenging in hyperspectral super-resolution (HSR). Although recent methods support arbitrary-scale reconstruction, applying them to new sensors or scales beyond the training range often requires additional data and computation to maintain reconstruction quality. To address these challenges, we propose OmniHSR, which predicts band-shared spatial operators rather than spectral values. Cross-Spectral Mapping (CSM) resamples inputs with any number of bands to fixed reference positions and predicts local operators with Gaussian supports. Continuous Operator-Field Reconstruction (COFR) composes these operators into a continuous field and applies them to all original bands for arbitrary-scale reconstruction. Experiments demonstrate that operator prediction outperforms direct spectral-value prediction on all seven datasets. Trained solely on ARAD with only 0.538M parameters, OmniHSR outperforms all directly transferred baselines on six unseen datasets without target-domain training data or adaptation. Across twelve upsampling factors from $\times2$ to $\times48$, it improves average PSNR on Pavia U and Chikusei by 0.55 dB over the strongest baseline. It also surpasses baselines trained from scratch or adapted on the target sensor and achieves up to $36\times$ faster inference. Our code will be publicly released soon.
1 INTRODUCTION
OmniHSR addresses the difficulty of combining cross-sensor generalization with arbitrary-scale reconstruction by predicting band-shared spatial operators. Its experiments report stronger reconstruction than spectral-value prediction and zero-shot transfer to unseen sensors.
1 INTRODUCTION
- HSR methods commonly target fixed upsampling factors, while differing sensor band counts and often single-scene public data make cross-sensor, arbitrary-scale reconstruction challenging.Methods that predict spectral values may require sliding-window inference, target-sensor data, or retraining when sensors or scales change.Source: §1 · p. 2
- OmniHSR predicts band-shared spatial operators rather than spectral values, so one network run can serve all bands of a sensor.The paper motivates this design by noting that all bands share the same spatial mixing of high-resolution spectra within an observation footprint.Source: §1 · p. 2
- CSM maps inputs with any band count to fixed reference positions and predicts local Gaussian-supported operators; COFR composes them into a continuous field applied to original bands at any factor.Together, the modules address the changing input band count and target-grid scale.Source: §1 · p. 2
- 0.55 dB higher average PSNR than the strongest baseline on Pavia U and Chikusei across twelve factors from ×2 to ×48; OmniHSR also beats transferred baselines on six unseen datasets.It surpasses baselines trained from scratch or adapted on target sensors and runs up to 36× faster.Source: §1 · p. 2
2 RELATED WORK
Prior hyperspectral super-resolution work spans fixed- and arbitrary-factor models, approaches to varying band counts, and continuous reconstruction or filter prediction. OmniHSR combines continuous spatial support with operators that act across every band of an unseen sensor.
2 RELATED WORK
- Single-image HSR has progressed through convolutional, attention, transformer, and state-space designs, while arbitrary-factor methods extend reconstruction beyond fixed scales.These networks are generally trained on one sensor at a time, with output channels tied to that sensor’s band count.Source: §2 · p. 3
- Approaches to varying band counts include spectral-axis denoisers, pretrained RGB networks, and remote-sensing foundation models, while fusion methods use a high-resolution guide image.The passage contrasts these approaches with OmniHSR, which needs no wavelength, guide image, or adaptation and transfers from 31 to up to 145 bands.Source: §2 · p. 3
- Continuous upsampling and Gaussian-rendering methods support arbitrary output grids, while prior filter-prediction methods apply predicted filters to input images or channels.OmniHSR gives each continuous-support Gaussian primitive a zero-sum operator predicted from the low-resolution image and applied to every band of an unseen sensor.Source: §2 · p. 3
3 PROPOSED METHOD
OmniHSR predicts band-shared spatial operators from a fixed-size spectral input, then composes them into a continuous operator field applied to the raw bands for arbitrary-scale reconstruction.
3 PROPOSED METHOD
- OmniHSR uses Cross-Spectral Mapping (CSM) to predict local spatial operators and Continuous Operator-Field Reconstruction (COFR) to apply them to the raw input at arbitrary scales.The unified model targets varying band configurations and arbitrary output scales.Source: §3 · pp. 3–4
3.1 BAND-SHARED OPERATOR PRIOR
- The band-shared operator prior uses one zero-sum spatial operator across bands when every band undergoes the same spatial degradation.The resulting combination preserves weights summing to one and reduces to bicubic interpolation when the operator is zero.Source: §3.1 · p. 4
3.2 CROSS-SPECTRAL MAPPING
- CSM interpolates each pixel’s spectrum onto M reference positions and predicts a Gaussian-supported 5 × 5 operator at every low-resolution pixel, with no network parameters tied to band count.The predicted operator coefficients are made zero-sum, so regions with equal pixel values remain unchanged.Source: §3.2 · pp. 4–5
3.3 CONTINUOUS OPERATOR-FIELD RECONSTRUCTION
- COFR blends primitive operators with normalized Gaussian weights into a continuous, band-independent operator field, samples it on the target grid, and applies it to every raw band.The target grid can be evaluated at arbitrary continuous positions; the resulting reconstruction weights sum to one.Source: §3.3 · pp. 5–6
4 EXPERIMENTS
OmniHSR is evaluated on seven datasets, including six test-only datasets from unseen sensors, using shared antialiased bicubic degradation. It leads the reported cross-sensor comparisons, including against target-trained baselines, while ablations test operator prediction and key design choices.
4.1 EXPERIMENTAL SETUP
- The evaluation trains OmniHSR on ARAD alone and tests six other datasets from five unseen sensors, while baselines use ARAD training and 31-band sliding windows or target-scene training.The datasets span 31 to 145 bands; low-resolution inputs use antialiased bicubic downsampling, and reported metrics are PSNR and SAM in degrees.Source: §4.1 · p. 7
4.2 COMPARISON ON UNSEEN SENSORS
- 0.32–2.07 dB PSNR and 0.10–0.40° SAM gains over the best baseline on all seven datasets at ×4 accompany a lead across twelve factors on Pavia U and Chikusei.On those two unseen sensors, the average PSNR gain across twelve factors is 0.55 dB; OmniHSR also remains above bicubic in both metrics on all seven datasets.Source: §4.2 · p. 8, Table 4 · p. 9
- On Pavia U and Chikusei test tiles, target-trained baselines remain 0.10–1.03 dB below zero-shot OmniHSR at all six factors from ×3.5 to ×12.Baselines were trained from scratch or through band adapters.Source: §4.2 · p. 8
- At ×4, OmniHSR recovers thin, continuous Chikusei roads and matches the ground-truth spectrum at a circled road pixel, while baselines blur roads and mix neighboring-field spectra.On Pavia U, the circled-pixel spectral RMSE is 0.017 for OmniHSR versus 0.069 for SRNO.Source: Figure 3 · p. 8, §4.2 · p. 9
4.3 ABLATION STUDY
- Removing COFR costs 0.53 dB on Pavia U and 1.09 dB on Chikusei; dropping the zero-sum constraint costs 0.12 and 0.27 dB, respectively.Direct spectral-value output falls below bicubic on both sensors, by 2.3 dB on Chikusei; using 31 resampled bands costs 0.02 and 0.16 dB.Source: §4.3 · p. 9, Table 3 · p. 9
- A zero-sum 5×5 operator fitted on four bands per window is at least as accurate on the other bands as per-band operators across all seven datasets at ×2 and ×4.Fitting on a single band loses at most 0.01 dB on six datasets and up to 0.29 dB on CAVE; band redefinition or gain/offset before versus after OmniHSR matches within 0.05 dB.Source: §4.3 · p. 9
5 CONCLUSION
OmniHSR’s cross-sensor performance depends on predicting band-shared spatial operators rather than spectral values. Its zero-shot transfer and extended-scale results are bounded by the shared-blur and registration assumption and by evaluation on synthesized low-resolution inputs.
5 CONCLUSION
- Operator prediction is more accurate than spectral-value prediction on every dataset, and target adaptation still leaves baselines less accurate than the unseen-sensor model.The ARAD-trained model transfers from 31 bands to sensors with up to 145 bands without target data and retains its lead far beyond its training range.Source: §5 · p. 9
ETHICS STATEMENT
The study uses public hyperspectral datasets under their original terms and reports no new data collection or human subjects.
ETHICS STATEMENT
- The work uses public datasets under their original terms, involves no new data collection or human subjects, and reports no known work-specific ethical concerns.Source: ETHICS STATEMENT · p. 10
A DATASETS AND IMPLEMENTATION DETAILS
The appendix describes the dataset protocol and implementation details available in the supplied passages, including the forward-pass outline and operator initialization. The evaluation uses synthesized low-resolution inputs and reports dataset-specific crop or tile sizes.
A DATASETS AND IMPLEMENTATION DETAILS
- The seven-dataset evaluation uses center 256×256 crops for close-range data and 64×64 tiles for remote-sensing scenes.Table 5 lists sensor, platform, band count, spectral range, and test size; Botswana retains 145 of 242 Hyperion bands after removing water-absorption and noisy bands, so wavelengths are unevenly spaced.Source: A DATASETS AND IMPLEMENTATION DETAILS · p. 15, Table 5 · p. 15
- Algorithm 1 takes a hyperspectral input X and factor s and outputs a reconstruction at dimensions (sh, sw, B), beginning with cross-band mapping and ending with bicubic upsampling.The forward-pass excerpt also shows a factor-conditioned feature map, one primitive per low-resolution pixel, and a zero-mean operator kernel.Source: A DATASETS AND IMPLEMENTATION DETAILS · p. 15
- The training loss combines Charbonnier reconstruction with degradation-consistency, spectral-angle, and spectral-curvature terms.The latter three terms have weights 0.1, 0.1, and 0.05, respectively; training uses one RTX 4090D and seed 2026.Source: A DATASETS AND IMPLEMENTATION DETAILS · p. 16
- The operator-prediction output layer is zero-initialized, so every operator starts at zero and the initial reconstruction operator is bicubic interpolation.The 5×5 window uses replicate padding at image borders.Source: A DATASETS AND IMPLEMENTATION DETAILS · p. 16
B.1 THE THREE EXISTING ROUTES
The section compares three ways to reuse baselines on a new sensor. Zero-shot OmniHSR exceeds the strongest route across the evaluated factors, while a sliding-window version trades additional inference cost for a small PSNR gain.
B.1 THE THREE EXISTING ROUTES
- The three routes are direct transfer with sliding windows, target-scene training from scratch, and adapter tuning of two band-conversion layers.The test uses non-overlapping 64×64 tiles; direct transfer uses no target-domain training, while scratch and adapter tuning use target-scene data.Source: §B.1 · p. 16, Table 6 · p. 17
- OmniHSR exceeds the strongest reuse route by 0.21 dB on Pavia U and 0.60 dB on Chikusei, and leads all eighteen configurations at every factor.The comparison averages PSNR over seven factors from ×3.5 to ×16.Source: §B.1 · p. 16, Figure 4 · p. 17
- At ×4, sliding-window OmniHSR raises PSNR from 28.94 to 29.12 dB on Pavia U and from 28.95 to 29.13 dB on Chikusei, at 3.6–3.8× latency.It requires 7–10 forward passes and at least 31 bands.Source: §B.1 · p. 16
B.2 ALL BASELINES ON SEVEN DATASETS
On seven datasets, the ARAD-trained OmniHSR is compared with arbitrary-scale baselines at multiple factors and on unseen scenes. It leads the baselines broadly, including on the three additional unseen crops at ×4.
B.2 ALL BASELINES ON SEVEN DATASETS
- OmniHSR leads every baseline in PSNR and SAM on every dataset at ×2, ×4, and ×8.At ×4, its PSNR gain over bicubic interpolation has a 95% bootstrap interval above zero on every dataset and factor; it also beats SRNO on every image and tile.Source: §B.2 · p. 17, Table 7 · p. 18
- Several baselines fall below bicubic interpolation: GaussianSR on all six unseen datasets at ×8, and every baseline on Botswana at ×4 and ×8.On Harvard and Botswana, every baseline also has higher SAM than bicubic interpolation at all three factors.Source: §B.2 · p. 17
- On the three additional unseen ×4 crops, OmniHSR has the highest PSNR and lowest SAM.The crops are selected by fixed rules that do not use any reconstruction.Source: §B.2 · p. 18, Figure 5 · p. 19
B.3 SSIM AND SAM
Across the merged results and ablations, OmniHSR maintains strong SSIM, SAM, and spectral error performance, including at ×24. The experiments also test band sharing, output choices, reference positions, and operator-window sizes.
B.3 SSIM AND SAM
- OmniHSR has the highest SSIM and lowest SAM on Chikusei and ARAD at all twelve factors; on Chikusei it also beats interpolation on both metrics throughout.At ×12 in the target-trained comparison, all methods and bicubic interpolation lie within 0.02 SSIM.Source: §B.3 · p. 19, Table 8 · p. 20, Table 9 · p. 20
- OmniHSR has the lowest per-band RMSE on five datasets and on 94% of ARAD bands and 84% of Harvard bands.Among the hardest bicubic-error edge pixels, its spectrum is closer to ground truth than SRNO’s at 76–90% of pixels.Source: §B.3 · p. 19, Figure 6 · p. 21
B.4 RECONSTRUCTIONS AT ×24
- At ×24, OmniHSR keeps building rows and a river bank sharper than the baselines, with the highest PSNR and lowest SAM on both crops.The 11×11-pixel input is outside the training range.Source: §B.4 · p. 22, Figure 7 · p. 22
C.1 TESTING THE BAND-SHARED OPERATOR PRIOR
- A shared operator fitted on four bands is at least as accurate as per-band operators on all seven datasets at ×2 and ×4.An operator fitted on the first half of the spectrum also serves the second half equally well except on CAVE; the tested one-operator-per-window class is coarser than OmniHSR’s operators.Source: §C.1 · p. 22, Table 11 · p. 23
C.2 OUTPUT PATH AND OPERATOR CONSTRAINT
- Direct spectral-value output falls below bicubic PSNR in 19 of 21 rows and OmniHSR in all 21; removing the zero-sum constraint leaves it below OmniHSR in 20 of 21 rows.The spectral-value output also has higher error than the operators in every band of all three illustrated crops.Source: §C.2 · pp. 23–24, Table 12 · p. 23, Figure 8 · p. 24
C.3 REFERENCE POSITIONS AND OPERATOR WINDOW
- From eight reference positions onward, PSNR on all three datasets stays within 0.1 dB of the 31-position model.With one position, ARAD and Chikusei lose 0.41 and 0.55 dB at ×4; operator windows from 7×7 to 9×9 remain within 0.09 dB of 5×5.Source: §C.3 · p. 24, Table 13 · p. 25, Table 14 · p. 25
C.4 TRAINING ON A DIFFERENT SOURCE SENSOR
The appendix tests whether OmniHSR’s behavior transfers across training sensors and altered inputs, then examines the predicted primitives and operators. The reported transfer, band-redefinition, blur, and operator-structure results define the scope of these tests.
C.4 TRAINING ON A DIFFERENT SOURCE SENSOR
- The Chikusei-trained model transfers in both directions, but gains less than the ARAD-trained model; it was trained on 234 tiles from one scene.It stays above bicubic interpolation except on Botswana at ×8, where it ties.Source: §C.4 · p. 25, Table 15 · p. 26
C.5 REDEFINING THE BANDS
- Redefining input bands changes predicted operators little and changes PSNR by at most 0.05 dB across the tested transforms and datasets.At ×4, the median relative operator-field change is at most 0.09 for subsets and merges, and 0.17 for gains and offsets.Source: §C.5 · p. 26, Table 16 · p. 27
C.6 BLUR THAT GROWS ALONG THE SPECTRUM
- With blur increasing along the spectrum, OmniHSR remains above SRNO on every image and tile by 0.49–1.78 dB on average at κ = 0.5.It also exceeds bicubic interpolation on every dataset and has the lowest SAM of the three methods.Source: §C.6 · p. 27, Table 17 · p. 27
C.7 TEST-TIME DEGRADATIONS
- The test-time degradation comparison evaluates inputs generated with Gaussian noise, isotropic blur, or s×s area averaging.The supplied table caption identifies the compared methods as bicubic interpolation, the best of six baselines, and OmniHSR.Source: §C.7 · p. 27, Table 18 · p. 28
C.8 WHAT THE NETWORK PREDICTS
- At edges, predicted supports stretch along the edge: the long axis aligns within 30° of edge direction for 98.5% of ARAD and 94.8% of Chikusei edge primitives.Operator norms are higher at edges than in flat regions, so the correction to bicubic interpolation is small away from edges.Source: §C.8 · pp. 27–28, Figure 9 · p. 29
D INFERENCE COST
OmniHSR maintains low inference cost as band count and magnification increase, while baselines may require repeated band-window passes or run out of memory at large factors.
D INFERENCE COST
- At ×4, OmniHSR latency rises from 11.8 ms at 31 bands to 22.3 ms at 145 bands, running 3.6–8.5 times faster than SRNO and up to 36.5 times faster than the slowest baseline.Baselines with more than 31 bands use 7–11 sliding-window passes, whereas OmniHSR runs once.Source: D INFERENCE COST · p. 29, Table 20 · p. 31
- On 31-band ARAD, methods take 9–33 ms at ×4 and OmniHSR takes 11.8 ms; from ×8 onward it is fastest, and at ×16 it is 2.7 times faster than the fastest baseline.Table 19 reports A40 latency at ×4 and ×16 across four band counts.Source: D INFERENCE COST · p. 29, Table 19 · p. 30
- Latency depends on input size and band count, so the three 31-band datasets share a runtime, as do Pavia C and Pavia U.Figure 10 times each magnification as an independent inference, using a 32 × 32 input and log-log axes.Source: D INFERENCE COST · pp. 29–30, Figure 10 · p. 30
- 613 ms at ×48 on 128-band Chikusei, 7.3–32 times faster than baselines reaching that factor.Meta-SR runs out of memory from ×12, while SRNO and GaussianSR do so at ×48.Source: D INFERENCE COST · p. 29, Figure 10 · p. 30