PaperlayerView exact revision on arXiv

Source-linked AI summary

VesperMUSE: Multi-Modal Fusion Vision Transformer for Image Semantic Segmentation Under Diverse Light Conditions

Raul Balmez

arXiv:2610.21990v1cs.CV

TL;DR

Low-light conditions degrade semantic segmentation, while CNN-based enhancement models have limited capacity to capture long-range spatial dependencies. VesperMUSE combines a primary modality with auxiliary modalities through a generalized fusion encoder. It reaches a peak mIoU of 0.7670 on CART, while higher computational cost is a trade-off.

  • Problem

    Low-light conditions degrade segmentation, and CNN-based models’ limited receptive fields constrain their capture of long-range spatial dependencies.

    Source: §2.2 · p. 20

  • Method

    VesperMUSE uses a fusion encoder that accommodates a primary modality and an arbitrary number of auxiliary modalities.

    Source: §3.4.1 · p. 40

  • Results

    VesperMUSE achieves a peak mIoU of 0.7670 on CART, with higher computational cost relative to baselines.

    Source: §5.1 · p. 72

  • Takeaways & Limitations

    For resource-constrained applications, CM-SSM is described as an optimal balance between competitive accuracy and a lower computational footprint.

    Source: §5.1 · p. 72

    A small number of synthetic low-light samples are excessively degraded, leaving information that the LLIE module cannot recover.

    Source: §5.1 · p. 70

Abstract

from arXiv · show

Semantic segmentation is critical for the reliable navigation and scene understanding of autonomous systems, yet its performance degrades severely in low-light environments due to noise, artifacts, colour distortion, and the loss of structural details. To address this challenge, this work introduces VesperMUSE, a unified multi-modal framework designed to achieve robust image semantic segmentation across diverse illumination conditions. The architecture addresses the light deficiency problem on multiple levels: proposing a novel physics-informed synthetic low-light adaptation algorithm, approaching Low-Light Image Enhancement (LLIE) through a thermal-guided paradigm, and finally aiming to recover the intrinsic scene details for semantic segmentation through a multi-modal fusion approach. Following enhancement, the framework dynamically integrates the primary RGB features with auxiliary thermal representations and surface normal maps highlighting geometric discontinuities and material boundaries. At the core of this integration is a highly scalable, modality-agnostic self-attention mechanism that extracts both global context and fine local features from an arbitrary number of auxiliary inputs. By combining and projecting these hierarchical multi-modal representations into a unified component, the attention module allows the network to selectively attend to relevant cross-modal cues without requiring any architectural modifications when different sensor configurations are used. Extensive evaluations demonstrate that VesperMUSE successfully recovers missing visual cues, providing a flexible, state-of-the-art (SOTA) solution for machine perception in extreme and unstructured real-world environments.

4.4 Qualitative comparison of VesperMUSE (VM) LLIE with SOTA methods enhancing

The supplied figures and tables cover class distributions, segmentation performance under different input configurations, LLIE comparisons, computational complexity, and modality and decoder ablations.

4.4 Qualitative comparison of VesperMUSE (VM) LLIE with SOTA methods enhancing

  • Figure 4.5 illustrates per-class surface distributions on PST900 and CART.The caption says each class percentage is computed as a mean percentage, but the supplied excerpt ends before specifying the denominator.Source: §4.4 · p. 4

4.7 Qualitative comparison of VesperMUSE (VM) and the LLIE configuration (VM LLIE)

  • Tables 4.8 and 4.10 compare segmentation performance using RGB, LLIE RGB, and unenhanced low-light RGB inputs.Their captions identify the input configurations but do not state the outcomes.Source: §4.7 · pp. 4–5
  • Tables 4.9 and 4.11 report segmentation-performance deltas between VesperMUSE LLIE and VesperMUSE trained and tested on low-light images.The captions frame the deltas as highlighting LLIE’s role but supply no values.Source: §4.7 · p. 5

4.2 Quantitative comparison of LLIE methods on the synthetically low-light adapted

  • Table 4.3 compares LLIE methods on the synthetically low-light-adapted PST900 dataset.The caption indicates best results but provides no values or winning method.Source: §4.2 · p. 6

4.4 Performance comparison with SOTA methods on the CART dataset [25]. Best results

  • Caption 4.5 describes a computational-complexity comparison at 480 × 640-pixel image resolution and indicates mIoU results, though the supplied passage ends before giving any values.Source: §4.4 · p. 6
  • Table 4.6 analyzes the impact of different primary and auxiliary input modalities on PST900 and CART.The caption says results are grouped by primary backbone.Source: §4.4 · p. 6
  • Table 4.7 ablates fusion and decoder strategies on PST900 and CART, including parameter counts and computational results.The supplied caption does not state which strategies perform best.Source: §4.4 · p. 6

Key terms and abbreviations

The section expands abbreviations used in the paper, covering image enhancement, image-quality measures, neural-network components, and related terminology.

Key terms and abbreviations

Declaration of originality

The author declares that no portion of the thesis has been submitted for another degree or qualification.

Declaration of originality

Copyright statement

The statement sets out the author’s copyright ownership and limits on copying and use of third-party intellectual property and reproductions.

Copyright statement

  • The author owns copyright and related rights in the thesis and grants the University of Manchester certain use rights, including for administrative purposes.Source: Copyright statement · p. 9
  • Thesis copies or extracts may be made only under the cited copyright law or applicable University licensing agreements, and must include this page.Source: Copyright statement · p. 9
  • Third-party intellectual property and reproductions described in the thesis require their owners’ prior written permission for use.Source: Copyright statement · p. 9
  • Further information on disclosure, publication, and commercialisation conditions is available through University policies and regulations.Source: Copyright statement · p. 9

2.1 Synthetic Low-Light Image Adaptation

VesperMUSE uses physics-informed, structure-aware synthetic low-light adaptation before thermal-guided enhancement and multimodal segmentation, matching real-world luminance statistics while adapting degradation to image brightness. The adaptation includes color reconstruction, optional exposure scaling, physically motivated sensor-noise simulation, and conditional hot-pixel cleanup.

2.1 Synthetic Low-Light Image Adaptation

  • Earlier LLIE data relied on RAW short- and long-exposure pairs, while RAW conversion could fail to reproduce natural low-light noise and illumination; later paired datasets include LOL and LOL-v2 real.The cited synthetic-data foundation also follows illumination distributions from multiple datasets and uses an accessible encoding format.Source: §2.1 · p. 19

2.2 Low-Light Image Enhancement

  • CNN-based LLIE can struggle with noise, artifacts, and long-range spatial dependencies, while ViT self-attention captures those dependencies to guide enhancement.The passages describe CNN receptive-field limits as especially restrictive because light interacts nonlinearly with scene regions.Source: §2.2 · p. 20

2.3 Modality Fusion for Image Semantic Segmentation

  • Multimodal segmentation supplements RGB with thermal or depth cues, and recent work is shifting from modality-specific systems toward unified architectures that integrate concurrent streams.Thermal imagery is described as less affected by changing illumination, while depth provides geometric and distance cues.Source: §2.3 · pp. 21–22

3.1 Framework Design and Motivation

  • VesperMUSE synthesizes low-light inputs, applies supervised thermal-guided enhancement, derives surface normals from monocular depth, and fuses RGB, thermal, and normal features for segmentation.The enhancement model predicts original RGB images, and each dataset’s selected enhancement checkpoint is chosen by PSNR; segmentation checkpoints are selected by mIoU.Source: §3.1 · p. 24

3.2.1 Synthetic low-light adaptation algorithm

  • The algorithm estimates real-world luminance statistics from LOL, LOL-v2 real, SDSD-in/out, and LOL-Blur, then matches illumination while retaining original reflectance-based structure.It derives luminance from the Y channel in YCbCr and uses a Gaussian-smoothed illumination estimate with σ = 15.Source: §3.2.1 · pp. 25–27
  • The synthesis adjusts degradation to an image’s existing luminance and blends reconstructions to limit impact on already-dark scenes while preserving structurally significant highlights.The blending uses a luminance threshold τ = 150, transition parameter κ = 100, and structure-bias parameter β ∈ [0, 1].Source: §3.2.1 · pp. 27–28
  • The synthetic RGB image is reconstructed by combining final luminance with the original Cr and Cb chrominance channels and applying the inverse color-space transform.Source: §3.2.1 · p. 28
  • An optional randomly sampled global dimming factor scales the reconstructed RGB image within an exposure range [smin, smax] to vary exposure patterns.Source: §3.2.1 · pp. 28–29
  • Noise simulation linearizes the gamma-compressed RGB image, adds Poisson shot noise and Gaussian read noise per channel, then returns the signal to sRGB.The model uses λ = 300 for shot noise, σ = 0.01 for read noise, and a tunable gain factor.Source: §3.2.1 · p. 29
  • A conditional 3 × 3 median filter replaces only pixels above intensity 250, preserving high-frequency structural details elsewhere.The filter is a final post-processing step for isolated extreme-intensity pixels produced by statistical noise simulation.Source: §3.2.1 · pp. 29–30

3.2.2 Auxiliary modalities

VesperMUSE uses thermal imagery and surface normals as complementary inputs to support low-light enhancement and segmentation. Its LLIE module processes RGB and thermal inputs through a hierarchical encoder–bottleneck–decoder architecture.

3.2.2 Auxiliary modalities

  • Thermal imagery contributes complementary information through material properties such as thermal inertia and emissivity, with its cues varying across PST900 and CART scenes.The passages describe thermal imaging as a structural cue in PST900 and as a way to discriminate outdoor textures in CART.Source: §3.2.2 · p. 30
  • Surface normals derived from monocular depth emphasize geometric discontinuities, helping distinguish object boundaries when enhanced RGB remains distorted and nearby objects appear thermally consistent.The scale-invariant geometric representation is described as useful for both close objects in PST900 and distant objects in CART.Source: §3.2.2 · p. 30

3.3 Low-Light Image Enhancement Vision Transformer

  • The LLIE architecture uses a U-shaped encoder–bottleneck–decoder, with hierarchical thermal-aware blocks integrating RGB features and scale-matched thermal representations.The encoder reduces spatial dimensions while increasing feature depth; the decoder upsamples and concatenates encoder features before refinement.Source: §3.3 · pp. 31–32
  • The LLIE module takes synthetic low-light RGB and corresponding thermal images and predicts an enhanced RGB image.The model is trained to predict the original RGB map using PST900 and CART data.Source: §3.3 · p. 31

3.3.1 Thermal-aware Transformer Block

The thermal-aware block combines RGB and thermal features through cross-attention, then refines the result with a feed-forward network. Thermal cues provide local and global structure while RGB supplies colour information.

3.3.1 Thermal-aware Transformer Block

  • The Thermal-aware Transformer Block combines RGB-Thermal Cross-Attention and a Feed-Forward Network, with layer normalization and residual skip connections.The block’s output depends on the input features and the corresponding thermal features at each level.Source: §3.3.1 · p. 33

3.3.2 RGB-Thermal Cross-Attention

  • Thermal maps provide local and global structure cues for enhancement, while the degraded RGB map supplies colour information.The attention module is intended to capture patterns that guide restoration of details lost under low-light conditions.Source: §3.3.2 · p. 33
  • RGB and thermal feature maps are projected into query, key, and value embeddings; multi-head attention scores are applied to thermal values and projected into an output feature map.Each head uses a learnable temperature parameter, and the attention output is refined by the feed-forward network.Source: §3.3.2 · pp. 33–34

3.3.3 Feed Forward Network

  • The feed-forward network refines the thermal-guided feature map using three convolutional layers with 1 × 1 and 3 × 3 kernels, interleaved with GELU activations.Source: §3.3.3 · p. 34

3.3.4 Low-Light Image Enhancement Loss function

The LLIE objective combines image fidelity, perceptual quality, structural consistency, and semantic preservation, while the segmentation model fuses hierarchical RGB, thermal, and surface-normal features. Its self-attention fusion is designed to scale to arbitrary auxiliary inputs without architectural changes.

3.3.4 Low-Light Image Enhancement Loss function

  • The LLIE objective combines MAE reconstruction, perceptual, structural-similarity, and semantic-consistency losses to support visual quality and downstream semantic preservation.The semantic term uses a frozen pretrained ResNet50 feature encoder as a high-level regularizer.Source: §3.3.4 · pp. 34, 36

3.3.5 Low-Light Image Enhancement Training

  • The LLIE module trains for 300K iterations with Adam on randomly cropped 256 × 256 paired low-light and normal-light patches, using batch size 8.Training uses a two-stage cosine schedule, starting at 2 × 10^-4, rising to 3 × 10^-4 by 92K iterations, then decreasing to 1 × 10^-6 over 208K iterations.Source: §3.3.5 · p. 36

3.4 Fusion Vision Transformer

  • Fusion self-attention integrates global and local patterns from auxiliary modalities and accommodates an arbitrary number of inputs without structural changes to the fusion method.The encoding pathway uses four cascaded Fusion Encoder Blocks, each applying fusion self-attention followed by a feed-forward network.Source: §3.4 · pp. 37–38
  • The segmentation system uses RGB as its primary modality and thermal imagery plus surface normals as auxiliary modalities, projecting auxiliary features to align their dimensions and learn a shared latent space.The primary stream uses ResNet50, while auxiliary streams use lighter ResNet18 backbones for computational efficiency.Source: §3.4 · pp. 37–39
  • Fusion occurs during encoding; decoding progressively refines the fused representation across three spatial levels using skip connections from corresponding encoding stages.The bottleneck is upsampled and concatenated with an encoder skip connection, and this process repeats three times before final upsampling.Source: §3.4 · pp. 39–40

3.4.1 Fusion Encoder Block

The Fusion Encoder Block combines and evaluates information from multiple modalities, while its self-attention module extracts both global context and fine local features. Its structure supports one primary modality and an arbitrary number of auxiliary modalities.

3.4.1 Fusion Encoder Block

  • The Fusion Encoder Block scales to an arbitrary number of modalities by designating one primary modality and treating the rest as auxiliary inputs.It combines modality features and assesses both their distinct information and their interactions.Source: §3.4.1 · p. 40

3.4.2 Fusion Self-Attention Module

  • The Fusion Self-Attention Module aims to extract both global and local features, adapting latent representations using a wider range of information.This addresses the need to retain local details such as edges alongside global context for semantic segmentation.Source: §3.4.2 · p. 41
  • The primary modality supplies the Query and Key representations, while auxiliary modalities provide local and global features combined into the Value representation.Local auxiliary features are concatenated and projected; global features are likewise pooled, concatenated, and projected before the two projections are multiplied.Source: §3.4.2 · pp. 41–42
  • The module splits Query, Key, and Value into heads to reduce complexity and improve contextual interaction, then projects the final map back to the original depth.Its attention formulation uses temperature T to control the balance of attention scores.Source: §3.4.2 · pp. 42–43

3.4.3 Feed Forward Network

The Fusion Vision Transformer uses a decoder to process fused features, a CNN segmentation head to produce class logits, and a weighted hybrid loss with deep supervision. The loss combines pixel-wise and region-based objectives to address class imbalance and structural consistency.

3.4.3 Feed Forward Network

  • The Fusion Vision Transformer reuses the LLIE module’s feed-forward network.The cited passage points to Section 3.3.3 for its formal description.Source: §3.4.3 · p. 43

3.4.4 Decoder Block

  • The decoder iteratively processes fused features concatenated with encoder skip connections, with its attention representations derived from the same input feature.Unlike the Fusion Encoder Block, the decoder derives Query, Key, and Value from one source.Source: §3.4.4 · pp. 43–44

3.4.5 Segmentation Head

  • The segmentation head maps fused features to class logits using two convolutional layers, batch normalization, ReLU, and an intervening dropout layer.Dropout is used as a stochastic regularizer to reduce feature co-adaptation and overfitting to high-dimensional noise.Source: §3.4.5 · p. 44

3.4.6 Fusion Vision Transformer Loss function

  • The total training objective uses a weighted hybrid loss at final and intermediate decoder predictions for deep supervision.The hybrid combines multi-class Focal loss with weighted Dice loss, balancing pixel-level classification and region-level overlap.Source: §3.4.6 · p. 45
  • Multi-class Focal loss down-weights well-classified pixels to address background-dominated class imbalance, while weighted Dice loss optimizes class-level overlap and supports small or weakly represented objects.Class-dependent weights further compensate for skewed class distributions.Source: §3.4.6 · pp. 45–46

3.4.7 Fusion Vision Transformer Training

VesperMUSE evaluates synthetic low-light adaptation, enhancement, and semantic segmentation on PST900 and CART. The reported results include strong segmentation benchmarks and competitive enhancement with a compact model, alongside synthetic adaptation metrics intended to reflect real low-light conditions.

3.4.7 Fusion Vision Transformer Training

  • PST900 contains 894 synchronized RGB-thermal pairs split 2:1 for training and evaluation, while CART contains 2,282 pairs split 6:1:1 across training, validation, and testing.The datasets cover subterranean and heterogeneous outdoor settings, respectively.Source: Experiments and Results · p. 49
  • Synthetic adaptation of PST900 and CART achieves approximately 10.7 dB SNR and a Gamma distribution of 1.00, surpassing LOL-v2-synthetic in the reported comparison.The authors describe the resulting data as retaining realistic luminance statistics and physics-informed effects of light deficiency.Source: Experiments and Results · p. 49
  • On synthetically adapted CART, VesperMUSE LLIE increases PSNR by 29.5% over LYT-Net and 2.3% over Retinexformer, with 0.454M parameters and performance comparable to ISALux.The reported parameter count represents a 75.4% reduction in model complexity relative to ISALux.Source: Experiments and Results · p. 49
  • On CART, VesperMUSE achieves mIoU 0.7672, exceeding CM-SSM (0.7460) by 2.8% and CMX (0.7400) by 3.7%.Reported class-wise results include Vehicles (0.6314) and Person (0.3426).Source: Experiments and Results · pp. 49–50
  • On PST900, VesperMUSE achieves mIoU 0.8559, improving over LASNet by 1.4% and MDNet by 3.1%.It also exceeds CM-SSM by 2.9% for Hand-Drill (0.8282) and 2.4% for Fire-Ext. (0.8776).Source: Experiments and Results · p. 50

4.1 Synthetic Low-Light Image Adaptation

  • The synthesis pipeline adjusts illumination to match a real-world luminance distribution after decomposing images into illumination and reflectance components.The evaluation considers exposure, gamma, luminance, variability, SNR, and structural information preservation.Source: §4.1 · p. 50

4.1.1 Quantitative Analysis

The section assesses synthetic low-light fidelity and thermal-assisted enhancement, then defines the metrics used to evaluate enhancement and semantic segmentation. Reported comparisons describe synthetic data as closer to real low-light statistics and VesperMUSE LLIE as competitive with larger enhancement methods.

4.1.1 Quantitative Analysis

  • The proposed synthetic datasets more closely match real datasets in mean luminance and exposure, while earlier synthetic approaches show Gamma values approximately 26% above real-world values.The proposed pipeline is reported to produce a perfectly linear mapping; the accompanying figure caption says it preserves signal quality similar to true low-light conditions despite prior methods’ excessive noise and over-brightening.Source: §4.1.1 · p. 53, Figure 4.1 · p. 53
  • The proposed low-light datasets share less information with their normal-light counterparts (0.31 and 0.29) than LOL-v2-synthetic (0.60).The authors frame this as more information needing recovery through enhancement.Source: §4.1.1 · p. 54

4.1.2 Qualitative Analysis

  • Synthetic samples show colour distortions, structured noise, and spatially and statistically variable illumination resembling real-world conditions.The cited qualitative assessment refers to PST900 and CART adapted samples.Source: §4.1.2 · p. 54, Figure 4.2 · p. 54, Figure 4.3 · p. 54

4.2 Low-Light Image Enhancement

  • PSNR evaluates pixel-level reconstruction error, while SSIM compares luminance, contrast, and structural information between images.Both metrics are used to assess reconstructed-image fidelity and visual quality.Source: §4.2 · p. 55

4.2.1 Quantitative Analysis

  • ISALux leads on both datasets at 27.58 dB/0.874 PSNR/SSIM on PST900 LL and 27.81 dB/0.873 on CART LL; VesperMUSE LLIE reports 27.47 dB/0.870 and 27.74 dB/0.858, respectively.VesperMUSE LLIE uses 0.454M parameters, about 4× fewer than the cited 1.85M comparison, with PSNR degradation of at most 0.11 dB.Source: Table 4.2 · p. 55, §4.2.1 · p. 56
  • VesperMUSE LLIE improves PSNR over Retinexformer by 0.87 dB on PST900 LL and 0.63 dB on CART LL, while using approximately 3.5× fewer parameters.Retinexformer has slightly higher SSIM on CART LL, with the passage describing the gap as small.Source: §4.2.1 · p. 56

4.2.2 Qualitative Analysis

  • Qualitatively, VesperMUSE LLIE improves local contrast and colour saturation and uses thermal cues to sharpen edges and suppress noise in low-light regions.The cited comparison reports clearer boundaries and stable textures without common artifacts such as halos or over-smoothing.Source: §4.2.2 · pp. 56–57

4.3 Semantic Segmentation

  • Semantic segmentation is evaluated primarily with mIoU, defined using per-class true positives, false positives, and false negatives.The passage identifies N as the total number of semantic classes.Source: §4.3 · p. 57

4.3.1 Quantitative Analysis

VesperMUSE reports strong segmentation results across PST900 and CART, while qualitative analyses, computational comparisons, and ablations examine its behavior and trade-offs. The studies show that enhancement and modality choice matter, with performance depending on dataset and class characteristics.

4.3.1 Quantitative Analysis

  • VesperMUSE reaches 0.8559 mIoU on PST900, 0.0031 below CM-SSM, while setting class bests for Hand-Drill (0.8282) and Fire-Extinguisher (0.8776).The reported class results exceed SA-Gate and CM-SSM, respectively.Source: §4.3.1 · p. 58, Table 4.3 · p. 58
  • VesperMUSE achieves 0.7672 mIoU on CART, ahead of CM-SSM (0.7460) and CMX (0.7400), with reported class leads on Bare Grd. and Road.It scores 0.8706 on Bare Grd. versus CM-SSM's 0.8260, and 0.6538 on Road versus CRM's 0.6160.Source: §4.3.1 · pp. 58–59, Table 4.4 · p. 59

4.3.2 Qualitative Analysis

  • Qualitative comparisons show VesperMUSE maintaining crisp predictions for targets in darkness and larger CART objects, while LLIE recovers some hidden structure but struggles with smaller classes.On CART, LLIE results are dominated by large classes such as Bare Ground, Rocky Terrain, and Water.Source: §4.3.2 · pp. 59–60

4.4 Computational Performance

  • On CART, VesperMUSE reports 0.767 mIoU, 2.1% above CM-SSM and 3.0% above MCNet-T, while using 180.45M rather than 199.79M parameters.The multi-modal design increases FLOPs; the tri-input framework and DepthAnything V2 depth estimation, at 43 GFLOPs, are identified as contributors.Source: §4.4 · p. 63

4.5.1 The effect of LLIE on Semantic Segmentation performance

  • On PST900, LLIE-related gains are concentrated in Fire-Ext. and Survivor, whereas Hand-Drill and Backpack show slight declines.The text links this pattern to differences in class colors and instance surface area, while noting that the latter explanation is a candidate.Source: §4.5.1 · pp. 63–64
  • On CART, unenhanced low-light RGB performs worse for every class, while enhancement preserves the dataset's more even natural-light color balance; Person remains absent from predictions.The passage attributes the persistent Person issue to the small surface covered by noise across its instances.Source: §4.5.1 · p. 64

4.5.2 Modality choices and their rank

  • Thermal and surface normals each contribute as auxiliaries; their combined benefit is strongest on PST900, with an average improvement of about 5% across both modalities.The reported average deficits are about 3% without surface normals and about 3.5% across both datasets when considering the auxiliary contribution.Source: §4.5.2 · pp. 65–66
  • Using surface normals as the main modality yields an average performance deficit of about 7%, and thermal as the main modality declines relative to configurations with LLIE RGB primary.The text concludes that color information remains more important for segmentation than clean thermal imaging in this ablation.Source: §4.5.2 · pp. 65–66

4.5.3 Architectural motivation

The architectural ablations favor cross-attention fusion and attention-based decoder refinement over simpler alternatives. The discussion also identifies limitations in synthetic low-light samples, enhanced-image colour, and computational cost, alongside future directions and dataset constraints.

4.5.3 Architectural motivation

  • Cross-attention fusion outperforms naive feature concatenation by approximately 4% mIoU.The comparison is reported for the proposed fusion approach against concatenation in Table 4.7.Source: §4.5.3 · p. 67, Table 4.7 · p. 68
  • Replacing the Transformer decoder blocks with pointwise convolutions lowers mIoU from 0.7818 to 0.7351 on PST900 and from 0.5957 to 0.5320 on CART.The convolutional substitution slightly reduces parameter count but increases FLOPs from 321G to 326G; the reported accuracy drop supports the role of spatially aware refinement.Source: §4.5.3 · p. 68, Table 4.7 · p. 68

5.1 Known Limitations

  • A small number of synthetically adapted low-light samples are so degraded that the LLIE module cannot recover their lost information, and their cause remains unknown.The authors suspect illumination variability; some affected samples are noise- and artifact-covered and cannot be correctly evaluated by segmentation.Source: §5.1 · p. 70
  • The LLIE module distorts colour: it attenuates blue in shadows and mid-tones, then exhibits green-channel clipping in extreme highlights.These changes disrupt intrinsic RGB ratios; the text notes that the current loss configuration lacks an integrated colour loss.Source: §5.1 · pp. 70–71
  • VesperMUSE reaches a peak mIoU of 0.7670 on CART, but has higher computational cost than baselines; CM-SSM offers competitive mIoU of approximately 0.7460 at lower cost.The discussion presents CM-SSM as an efficient alternative for resource-constrained applications.Source: §5.1 · pp. 71–72

5.2 Future Work

  • Future work proposes more efficient backbones, modality-specific backbones, alternative image representations, and low-/high-frequency decomposition in LLIE.The authors also suggest reducing decoder projection sizes, modality-based pruning, mixed precision, or quantisation as possible ways to reduce computational demands.Source: §5.1 · pp. 72–73, §5.2 · p. 73

5.3 Final Thoughts

  • The framework is intended as a foundation for segmentation under variable illumination, while limited availability of diverse, representative datasets constrains robust and generalizable method development.The discussion also emphasizes transparency about limitations in safety-sensitive settings.Source: §5.3 · p. 73

Related papers

Loading 2610.21990v1…