PaperlayerView exact revision on arXiv

Source-linked AI summary

SPECTRA-VLA: Base-Fine Hyperspectral Token Transfer at Frozen Vision-Language-Action Interfaces

Chih-Chung Hsu

arXiv:2610.19651v1cs.CV

TL;DR

RGB-similar objects can differ spectrally, but replacing a VLA’s visual representation risks losing token structure its frozen decoder expects. SPECTRA-VLA adds a continuous spectral residual through each target token’s own reference without retraining the encoder or decoder. Across 86 held-out acquisitions, category bAcc rises from 0.812 to 0.883, alongside improved receiving-token reuse and within-category spectral recovery.

  • Problem

    RGB-similar waste objects can have different spectra, while replacing the visual representation can discard token structure expected by a pretrained decoder.

    Source: §1 · p. 1

  • Method

    SPECTRA-VLA adds a learned continuous spectral residual through each target token’s own reference at native width while keeping the VLA components fixed.

    Source: §1 · p. 1

  • Results

    Category bAcc rises from 0.812 to 0.883 across 86 held-out acquisitions, and own-token composition approximately doubles target-specific recovery increments after frozen decoding.

    Source: §1 · p. 1, §8 · p. 10

  • Takeaways & Limitations

    Spectral enrichment and receiving-token preservation work together at the pretrained interface, while continuous input retains within-category spectral variation.

    Source: §7 · p. 9, §8 · p. 10

    The four-state target is a category-derived proxy, not an independently labeled robot action, and the object categories are not verified physical-composition identities.

    Source: §3 · p. 3

Abstract

from arXiv · show

Hyperspectral imaging distinguishes RGB-similar objects, but a pretrained vision-language-action model needs a state its frozen decoder can use. SPECTRA-VLA provides a native-width spectral upgrade without retraining the visual encoder or action decoder. A learned continuous residual enriches each target token through its own normalized false-color reference; projection, norm control and validation-calibrated injection regulate the change. Across 86 held-out acquisitions, category balanced accuracy rises from 0.812 to 0.883. With identical residuals and scale, own-token addition improves reuse of an external category-derived probe on frozen decoder logits: 0.701 versus 0.590 with pooled replacement, at similar image-disjoint recognition. Five grouped tests covering the same 86 acquisitions recover within-category spectral shape with $R^2=0.802/0.337$ from continuous and posterior-code states. Category-matched spectral swaps further show that own-token composition approximately doubles the target-specific recovery increment after frozen decoding. Spectral enrichment and receiving-token preservation thus work together at the pretrained interface.

1 INTRODUCTION

SPECTRA-VLA adds continuous spectral information at the frozen VLA interface while preserving each receiving token’s reference. It addresses the tension between distinguishing RGB-similar objects and retaining token structure usable by a pretrained decoder.

4 BASE-FINE HYPERSPECTRAL TRANSFER

  • A learned residual enters through each target token’s own reference at native width, with projection, norm control, and validation-calibrated injection regulating the change.The training heads are discarded while both VLA components remain frozen.Source: §1 · p. 1, §2.2 · p. 2, §4 · p. 3

1 INTRODUCTION

  • Category bAcc rises from 0.812 to 0.883 across 86 held-out acquisitions with receiving-token-preserving spectral transfer.A second backbone reproduces fixed-split enrichment (0.812 to 0.914).Source: §1 · p. 1

4.2 Spectral Fine Residual

  • Across five session-grouped tests, continuous and posterior-code states recover within-category spectral shape with R2 = 0.802/0.337.The fine branch pairs raw spectral bands with NMF coordinates; projection defines a base-relative insertion direction, and the largest searched scale sets a pre-composition residual budget of 0.4 of the base norm.Source: §1 · p. 1, §4.2 · p. 3

4.3 Composition, Endpoint, and Geometry

  • Across 86 acquisitions, own-token addition approximately doubles target-specific recovery increments after frozen decoding, and its addition branch retains the receiving token’s own reference.Matched equal-movement comparisons attribute smaller disturbance tails to the selected operating point.Source: §1 · p. 1, §4.3 · p. 4

2.1 VLA Models and Beyond-RGB Sensing

  • SPECTRA-VLA keeps both the visual encoder and action decoder fixed, unlike cited beyond-RGB approaches that train downstream VLA components with demonstrations.The paper also distinguishes its unchanged downstream consumer from tactile VLA approaches that condition or modulate visual features.Source: §2.1 · p. 2

2.2 Spectral Representations and RGB Transfer

  • SPECTRA-VLA inserts category-supervised continuous spectral content at the existing visual-token connection while composition preserves each receiving token’s reference.Related approaches include learned spectral representations, adapters, and token replacement; SPECTRA-VLA retains token content, not only position.Source: §2.2 · p. 2

3 PROBLEM SETTING AND EVIDENCE SCOPE

  • The reference is a fixed HSI-derived SWIR false-color view, and the four-state target is a category-derived proxy rather than an independently labeled robot action.The six object labels are operational categories, not verified physical-composition identities.Source: §3 · p. 3

4.1 False-Color Reference and Target Support

  • The normalized false-color reference grid provides the base-relative geometry and common token scale; zero injection exactly restores that reference.The operational reference changes the interface despite frozen model parameters.Source: §4.1 · p. 3

5 EXPERIMENTAL PROTOCOL

The evaluation combines an image-disjoint fixed split with a principal session-held-out assessment. Controls compare spectral input, composition, and decoder readouts under defined data splits and inference configurations.

5.1 Data and Split Boundaries

  • The principal evaluation holds out each of 86 acquisition sessions, while the fixed split is image-disjoint but not session-disjoint.Fitted transforms, candidate settings, thresholds, and readouts are estimated inside outer training.Source: §5.1 · p. 4

5.2 Comparison Controls and Readouts

  • The controls compare replacement, addition, and residual constraints using matched training settings and category and decoder-logit readouts.Pooled replacement uses the same residual and injection scale as own-token addition, but omits the receiving token’s own reference from its injected branch.Source: §5.2 · pp. 4–5

5.3 Frozen Decoder and Uncertainty

  • OpenVLA receives the full token grid and a generic sorting prompt; decoder comparisons jointly recompute reference and altered states in one inference configuration.Three-seed spread describes optimization rather than sampling uncertainty, and the stress-pair set contains 27 test pairs over 49 images and 38 sessions.Source: §5.3 · p. 5

6 RESULTS

Across 86 held-out sessions, SPECTRA-VLA improves category recognition while retaining spectral information and supporting reuse of a frozen decoder readout. The results also show that own-token composition, injection calibration, checkpoint choice, and target quality shape measured performance.

6.1 Session-Held-Out Category Transfer

  • Category bAcc improves by 0.071 (95% CI [0.050, 0.107]) across held-out sessions; base-fine reaches 0.883 versus 0.812 for the reference.The comparison uses fold-local selection and refitting followed by one evaluation per session.Source: §6.1 · p. 5, §6 · p. 5, Table 1 · p. 6

6.2 Spectral Content Before and After Decoding

  • Continuous states retain more within-category spectral shape than posterior codes (ΔR2 = 0.465 [0.429, 0.502]); after frozen decoding, continuous/posterior recovery is 0.096/0.085 versus 0.083 for reference.The post-decoding continuous-minus-posterior gain is 0.0113 (66-session CI [0.0042, 0.0189]).Source: §6.2 · p. 6, Table 2 · p. 7

6.3 Composition at a Frozen Interface

  • With residuals and scale fixed, own-token addition improves reference-trained decoder-probe bAcc to 0.701 versus 0.590 for pooled replacement at similar recognition.Replacement moves the pool less but has higher decoder-KL P95, 5.593 versus 1.158.Source: §6.3 · p. 7, Table 3 · p. 8
  • Own-token addition approximately doubles the target-specific recovery increment: 0.0336 versus 0.0165 for replacement, with interaction 0.0171 [0.0121, 0.0228].Category-matched swaps test the target-specific effect after decoding.Source: §6.3 · p. 7, Table 3 · p. 8

6.4 Calibrating Recognition and Disturbance

  • At a common movement allowance, base-fine matches unprojected recognition at 0.880 while reducing displacement and decoder-KL P95 to 0.188/0.152 from 0.213/0.182.At matched achieved movement, recognition and KL tails are similar, separating operating-point calibration from projection alone.Source: §6.4 · p. 7, Table 4 · p. 8

6.5 Checkpoint and Readout Sensitivity

  • At epoch 60, validation-selected q = (2, 4, 4) raises category/pair scores to 0.905/0.778 from 0.801/0.593, while reference-fixed bAcc falls by 0.077.The pair-gain session interval includes zero; earlier CE checkpoints with q = 4 give 0.701 in the same runtime/readout.Source: Figure 2 · p. 6, §6.5 · p. 8

6.6 Backbone, Grounding, and Failure Analysis

  • On the fixed image-disjoint split, SpatialVLA raises category/proxy bAcc from 0.812/0.871 to 0.914/0.947, but reference-fixed decoder-probe bAcc remains below reference, 0.673 versus 0.718.The paired session intervals for the gains are [0.043, 0.177] and [0.025, 0.127].Source: §6.6 · p. 8
  • Poor class-oracle predicted masks with mean IoU 0.218 lower base-fine category bAcc below the base, 0.362 versus 0.468, even with category supplied.A four-pixel shift gives 0.638/0.643 base-fine/base bAcc; one-pixel erosion gives 0.732/0.683.Source: §6.6 · p. 8

7 DISCUSSION

SPECTRA-VLA combines continuous spectral enrichment with receiving-token preservation, and the discussion presents these as complementary. Its tests distinguish content, composition, and calibration choices at the frozen interface.

7 DISCUSSION

  • Receiving-token controls tie decoder stability to grid structure rather than pooled proximity.The discussion describes spectral enrichment and interface preservation as complementary.Source: §7 · p. 9
  • Same-category spectral swaps show own-token addition approximately doubles the target-specific recovery increment after frozen decoding.The comparison separates content-dependent increments from the no-injection grid benefit.Source: §7 · p. 9
  • Continuous inputs retain within-category spectral shape, while posterior codes favor classification; learned residuals improve recognition and mean decoder KL over fixed maps, with a recovery trade-off.Projection sets a base-complement direction, while norm control and validation calibration regulate executed movement.Source: §7 · p. 9
  • SPECTRA-VLA preserves receiving tokens at native width while freezing both pretrained components; its auxiliary heads serve training only.The discussion contrasts this interface approach with HyA-T, which also freezes both components.Source: §7 · p. 9

8 CONCLUSION

SPECTRA-VLA upgrades frozen VLA visual-token interfaces at native width, preserving the pretrained encoder and decoder. Across the reported tests, continuous spectral input retains within-category variation, and own-token composition improves decoder-probe reuse and target-specific recovery.

8 CONCLUSION

  • Category bAcc rises from 0.812 to 0.883 across 86 held-out acquisitions, and a second backbone reproduces enrichment.The frozen encoder and decoder and the exact zero-injection reference are preserved.Source: §8 · pp. 9–10
  • Continuous input retains within-category spectral variation; own-token composition improves fixed-readout reuse at similar recognition and approximately doubles target-specific recovery increments.These results couple new spectral content to the pretrained receiving structure.Source: §8 · p. 10
  • Code and grouped predictions, donor identities, and numerical results are available online.The supplement specifies checkpoints and inference.Source: §8 · p. 10

S1 EXPERIMENTAL DETAILS

The supplement details the observations, training-fitted spectral features, adapter and token composition, selection procedure, readouts, and numerical settings used in the experiments. Evaluation distinguishes image holdout from acquisition-session holdout.

S1 EXPERIMENTAL DETAILS

  • Experiments distinguish spectral content, receiving-token composition, and injection calibration using pooled-state readouts and frozen decoder-logit probes.The supplement also acknowledges AI assistance with implementation, analysis, and preparation.Source: S1 EXPERIMENTAL DETAILS · p. 12

S1.1 Observations, Targets and Partitions

  • SpectralWaste targets are connected components in resized masks; session holdout separates all images acquired together, unlike the image-disjoint split.The data include synchronized RGB and 224-band HSI; six categories are also mapped to a fixed four-state proxy.Source: S1.1 Observations, Targets and… · p. 12, Table S1 · p. 12

S1.2 Training-Fitted Spectral Features

  • The 136-D spectral input combines 120 raw bands with 16 NMF abundances, with transforms fitted within each training partition.The release lacks a per-band calibration vector, so wavelength-dependent analyses use an approximate uniform grid.Source: S1.2 Training-Fitted Spectral Features · p. 12

S1.3 Adapter, Loss and Token Composition

  • The adapter trains on normalized base-plus-residual states and evaluates a full token grid, with fractional mask weights controlling tokenwise injection.Training samples injection scales, uses shared and residual heads, and discards auxiliary heads after training.Source: S1.3 Adapter, Loss and Token Composition · p. 13

S1.4 Optimization and Fixed-Split Selection

  • Matched learned representations use three seeds and 60 epochs; mechanism comparisons select checkpoints by validation composed-head loss before selecting scale.Candidate scales and readout regularization are selected using validation criteria.Source: S1.4 Optimization and Fixed-Split… · p. 13

S1.5 Pooled-State and Decoder Readouts

  • External four-state decoder-logit probes distinguish refitted accessible information from reuse of a reference-trained decision function.The setup also defines pooled-state readouts and reports token cosine, displacement, and decoder KL diagnostics.Source: S1.5 Pooled-State and Decoder Readouts · pp. 13–14

S1.6 Numerical Settings

  • OpenVLA numerical results depend on runtime and cache settings; the reported configurations differ in reference bAcc, while retained logits and predictions support reproduction.Eight-observation maximum and mean absolute logit differences are 0.40625 and 0.077215, with top-1 symbols agreeing.Source: S1.6 Numerical Settings · p. 14

S2 SPECTRAL CONTENT

The spectral-content studies compare correct and altered spectra, continuous inputs and category codes, and injection scales across fixed-split and session-disjoint tests. They report within-category shape recovery and an interaction between spectral correspondence and receiving-token composition.

S2.1 Correspondence and Category-Code Controls

  • Correct spectra outperform wrong-target, sample-permuted, and class-mismatched inputs in the jointly recomputed decoder comparison.Norm-random and band-permuted means are lower, but their simultaneous intervals cross zero.Source: S2.1 Correspondence and Category-Code… · p. 14
  • Category-code controls show small category/proxy differences, while posterior codes favor classification and smaller disturbance and continuous spectra favor shape retention.All continuous-versus-code paired intervals for category differences span zero.Source: S2.1 Correspondence and Category-Code… · p. 14

S2.2 Within-Category Spectral-Shape Recovery

  • Continuous-minus-posterior recovery is 0.450000, with session interval [0.397758, 0.504428].Spectrum-input, category-only, and shuffled-target controls give R² values of approximately 1, 0, and −0.184459, respectively.Source: S2.2 Within-Category Spectral-Shape… · p. 14

S2.3 Spectral-Recovery Calibration

  • At the primary recovery calibration point, test R² is 0.801, category bAcc 0.871, displacement P95 0.146, and decoder-KL P95 0.087.The secondary test recovery is 0.713095; scale can be chosen for an information requirement rather than maximum recognition.Source: S2.3 Spectral-Recovery Calibration · pp. 14–15

S2.4 Session-Disjoint Mechanism Tests

  • Continuous input preserves more within-category variation than posterior codes, with paired R² gain 0.465 [0.429, 0.502]; posterior codes favor classification by 0.021 [0.009, 0.034].The five session-disjoint groups hold out all 86 eligible sessions once.Source: S2.4 Session-Disjoint Mechanism Tests · pp. 15, 17
  • At similar category recognition, addition improves refitted decoder bAcc by 0.035 [0.015, 0.066].The reported gain persists when the first group’s six cardboard objects are removed and across every session deletion.Source: S2.4 Session-Disjoint Mechanism Tests · p. 17, Table S5 · p. 17
  • 0.0171 [0.0121, 0.0228] is the addition-versus-replacement interaction in target-specific recovery across five acquisition groups.Own/donor recovery increments are 0.0336 for addition and 0.0165 for replacement; same-acquisition donors retain an interaction of 0.0162 [0.0122, 0.0210].Source: S2.4 Session-Disjoint Mechanism Tests · pp. 15, 17

S3 COMPOSITION AND CALIBRATION

S3 tests how spectral residuals are aligned, composed with receiving tokens, and calibrated at the frozen interface. The comparisons distinguish the effects of residual content, token composition, and movement constraints.

S3.1 Alignment and Native-Width Composition

  • 0.977/0.813/0.605/0.852 versus 0.939/0.905/0.778/0.938 for cosine/category/pair/proxy, respectively, compares base-conditioned direct with base-fine.Reference-fixed decoder bAcc was 0.590/0.620 and KL was 2.029/1.400; these results characterize the tested architecture, alignment grid, and validation-high-anchor selection.Source: S3.1 Alignment and Native-Width… · p. 18

S3.2 Identical-Residual and Matched Ablations

  • 0.700684 versus 0.589521 reference-fixed decoder bAcc favors own-token addition over pooled replacement with identical residuals, checkpoints, and scales.Category/proxy bAcc was 0.896720/0.892346 versus 0.930654/0.924114, respectively; the comparison used a common recomputed decoder configuration.Source: S3.2 Identical-Residual and Matched… · p. 18, Table S8 · p. 20
  • 0.013432 [−0.014300, 0.058071] is the injection-by-composition interaction, whose interval spans zero despite addition advantages with and without injection.The fixed-readout addition advantage was 0.097731 without injection and 0.111163 with injection.Source: S3.2 Identical-Residual and Matched… · p. 18, Table S9 · p. 21

S3.3 Achieved Movement and Readout Sensitivity

  • 0.147383/0.147391 achieved movement and 0.870742/0.871170 category bAcc compare base-fine with no-projection at the matched 0.15 target.Mean/P95 KL was 0.057333/0.163369 versus 0.055741/0.160344; the matched-movement test separates projection’s attribution role from calibration of the achieved state.Source: S3.3 Achieved Movement and Readout… · p. 18

S3.4 Residual Constraints and Budget Sensitivity

  • 0.655 cosine and 7.148 KL follow removal of the norm bound; removing orthogonalization can improve recognition while retaining norm control.Projection defines a base-complement attribution, while norm and scale control the magnitude of its disturbance.Source: S3.4 Residual Constraints and Budget… · p. 19
  • 0.705 versus 0.673 reference-fixed probe bAcc favors selected constraints over fixed orthogonal ρ = 0.1 in the separate nested study.Orthogonal variants were selected in 77/86 folds; category/proxy contrasts and contrasts with unprojected ρ = 0.1 spanned zero.Source: S3.4 Residual Constraints and Budget… · pp. 19, 21

S4 SESSION-HELD-OUT GENERALIZATION

S4 evaluates recognition on held-out sessions, selecting and refitting preprocessing and candidate models without using each held-out session. It also checks how the result changes under utility and constraint sensitivity analyses.

S4 SESSION-HELD-OUT GENERALIZATION

  • Each of 86 outer folds holds out a complete session, while inner training selects candidates and outer training refits preprocessing and models.Every held-out object contributes one prediction; intervals condition on realized fits rather than repeating training and selection.Source: S4 SESSION-HELD-OUT GENERALIZATION · p. 21
  • 0.812 to 0.883 pooled category bAcc is the base-fine gain, with paired session gain interval [0.050, 0.107].Deleting each session from fixed predictions leaves gains of 0.0680–0.0779 over base and 0.0651–0.0769 over direct, conditional on the trained folds.Source: S4 SESSION-HELD-OUT GENERALIZATION · p. 21
  • 0.881547 versus 0.882859 category bAcc follows removal of the pair utility term, a difference of −0.001313 [−0.006327, 0.001768].The changed-choice folds were refit; the other 75 predictions remained fixed.Source: S4 SESSION-HELD-OUT GENERALIZATION · pp. 21, 23

S5 AMBIGUITY AND CLUSTERED INFERENCE

S5 defines cross-category stress pairs from RGB-similar objects and evaluates pair resolution with clustered uncertainty. It also examines how spectral gain relates to RGB similarity and how the ambiguity threshold affects the eligible inventory.

S5 AMBIGUITY AND CLUSTERED INFERENCE

  • The fixed-split threshold selection yields 27 test pairs at 0.75, while 0.85–0.95 yield no eligible test pairs for every method.The 27-pair test inventory contains 54 objects, 49 images, and 38 sessions; it is an object-disjoint clustered stress sample.Source: S5 AMBIGUITY AND CLUSTERED INFERENCE · pp. 23, 25, Figure S4 · p. 25
  • +0.185 is the base-fine-minus-base pair-resolution point estimate, with session interval [−0.030, 0.443].The session interval spans zero; category/session evidence provides the supported recognition gain.Source: S5 AMBIGUITY AND CLUSTERED INFERENCE · p. 23
  • 215 disjoint cross-category pairs show Spearman association 0.115 [−0.073, 0.308] between RGB similarity and spectral gain.Quartile means are positive but nonmonotone, and this sensitivity analysis does not select the central stress inventory.Source: S5 AMBIGUITY AND CLUSTERED INFERENCE · p. 25

S6 DECODER-READOUT DIAGNOSTICS

S6 tests decoder-readout reuse on fixed-split and session-held-out data, distinguishing fixed decision functions from representation-specific refits. It also examines session shift and sensitivity to normalization and action-coordinate choices.

S6.1 Fixed-Split Recognition and Decision-Function Reuse

  • 0.701 versus 0.704 refitted bAcc contrasts base-fine with base, while reference-fixed bAcc is 0.620 and differs by −0.084 [−0.137, −0.030].The 0.701 result uses earlier CE-selected epochs and q = 4; checkpoint and scale differ from the separately recomputed sensitivity.Source: S6.1 Fixed-Split Recognition and… · p. 25

S6.2 Session-Shift Diagnostic

S6.3 Normalization and Action-Coordinate Sensitivity

  • 0.730378 to 0.647454 after normalization and 0.610655 after injection are raw-trained readout bAcc values across states.The total raw-to-injected interval is [−0.205931, −0.006330]; individual differences span zero.Source: S6.3 Normalization and… · p. 26

S7 NUMERICAL GEOMETRY AND FORWARD EQUIVALENCE

The supplement checks executed-token geometry, training-surrogate behavior, and equivalence of the official forward with token reinsertion. The normalized false-color reference is an exact zero-injection endpoint, while positive-scale movement and decoder effects are evaluated separately.

S7.1 Executed-Token Geometry

  • Across 6,885 object states, maximum absolute normalized dot product is 2.49×10−7 and maximum norm-ratio error from ρ = 0.1 is 1.13 × 10−6.The normalized false-color reference G is the exact q = 0 endpoint, with zero token/logit differences and KL, top-1 agreement one, and zero downstream differences.Source: S7.1 Executed-Token Geometry · p. 26
  • Token alignment and occupancy jointly determine executed movement, while decoder KL is measured empirically.The epoch-60 geometry and validation-occupancy quartile results are distinct from the auxiliary category-head evaluation at validation-CE epochs.Source: S7.1 Executed-Token Geometry · p. 26, S7.2 Training Surrogate and Executed… · p. 26

S7.2 Training Surrogate and Executed Pool

  • At epoch 60, q = (2, 4, 4) yields cosine 0.9977 and mean/P95 relative distance 0.0569/0.1328 across 1,383 training/executed-pool comparisons.Protocol normalization also changes raw decoder response: raw-to-G action KL mean/P95/P99 is 2.610/5.413/6.888, with top-1 agreement 0.163.Source: S7.2 Training Surrogate and Executed… · p. 26

S7.3 Official Forward and Token Reinsertion

  • On 171 held-out RGB observations, official OpenVLA forward and same-grid extraction/reinsertion produce zero logit difference and KL, with top-1 agreements of one.This validates the visual insertion interface before normalization; the protocol-normalized false-color reference is the method’s zero endpoint.Source: S7.3 Official Forward and Token… · p. 28

S8 BACKBONE AND OBSERVATION ROBUSTNESS

The supplement tests transfer to a second frozen VLA, region perturbations, spectral-input and rendering variants, and conditional category recognition. Outcomes distinguish recognition gains from decoder-readout behavior and grounding failures.

S8.1 Second Frozen VLA

  • Base-fine reaches pooled cosine 0.985 and category/proxy bAcc 0.914/0.947 versus base 0.812/0.871 on SpatialVLA, while reference-fixed decoder performance remains lower.The experiment repeats the objects, train-only features, rank, seeds, epochs, and validation scale/ridge selection.Source: S8.1 Second Frozen VLA · p. 28

S8.3 Capacity and Spectral Bands

  • The full-input category bAcc is 0.886 versus base-band 0.835, while band exclusion retains 0.878 under fixed rank-64 hyperparameters.Raw-only rank 64 gives category/pair 0.894/0.796, versus 0.884/0.778 at rank 256.Source: S8.3 Capacity and Spectral Bands · p. 28

S8.2 Grounding

  • Ground-truth-mask erosion preserves base-fine/base bAcc 0.732/0.683, whereas largest predicted components supplied with the target category yield IoU 0.218 and bAcc 0.362/0.468.A four-pixel shift is approximately neutral at 0.638/0.643, showing that moderate region errors and poor target selection have different effects.Source: S8.2 Grounding · p. 28, Figure S5 · p. 29

S8.5 Class Errors and Conditional Recognition

  • Base-fine retains 0.975 of correct predictions and corrects 0.542 of errors at q = (2, 4, 4) and category ridge 100.The largest predicted target-category component example has IoU 0.0029; outcome-selected examples illustrate failure types rather than prevalence.Source: S8.5 Class Errors and Conditional… · p. 29

S9 PUBLISHED SPECTRAL FEATURES AND EXTERNAL MATERIALS

The supplement evaluates frozen spectral encoders and dataset-specific material-transfer features, while contrasting their evidence with the target-grounded VLA interface setting. The material tests use expanded features and distinct split protocols rather than native-width VLA composition.

S9.2 Dataset-Specific Material Transfer

  • On Wood, base-fine reaches bAcc 0.844 versus 0.849 for full-spectrum mean+SD; on SpecTex, accuracy is 0.317 versus 0.079 for three-band base and 0.314 for full-spectrum ridge.SpecTex is within-source interpolation with source arrays spanning all partitions; these are expanded ridge/PCA features, separate from native-width VLA composition.Source: S9.2 Dataset-Specific Material Transfer · pp. 30, 32

S9.3 Sensor and Supervision Coverage

  • The compared resources differ in supervision and annotation: SpectralWaste pairs imaging HSI/RGB, masks, and categories, while several driving datasets provide scene segmentation and many VLA resources lack imaging HSI.These differences motivate target-grounded spectral-interface evaluation, distinct from trajectory-supervised policy evaluation.Source: S9.3 Sensor and Supervision Coverage · p. 32

S9.1 Frozen Spectral Encoders

  • Published frozen spectral features are compared on the same 461 objects using common training-only readouts, but partial-load diagnostics are kept in distinct groups and not ranked.The compared feature sources include HyperSL, MaskedSST, SpectralGPT, DOFA, and HyperSIGMA; HyperSIGMA lacks pretrained parameters in active attention blocks.Source: S9.1 Frozen Spectral Encoders · pp. 29–30, Table S28 · p. 33

Related papers

Loading 2610.19651v1…