PaperlayerView exact revision on arXiv

Source-linked AI summary

Implicit Neural Representation for Cooperative Low-light Image Enhancement

Shuzhou Yang, Moxuan Ding, Yanmin Wu, Zihan Li, Jian Zhang

arXiv:2303.11722v3cs.CV

TL;DR

Low-light enhancement is limited by unpredictable degradation, a mismatch between metric-oriented and visual-friendly results, and scarce paired data. NeRCo uses controllable neural representation, vision-language supervision, and dual-closed-loop cooperative training to recover visual-friendly results unsupervised. The authors report robustness and superior effectiveness, including performance against a rich set of state-of-the-art algorithms and some supervised methods.

  • Problem

    Low-light enhancement is constrained by unpredictable brightness degradation and noise, a gap between metric-favorable and visual-friendly results, and limited paired training data.

  • Method

    NeRCo combines controllable neural representation normalization, vision-language-guided text-driven appearance discrimination, and dual-closed-loop cooperative adversarial training.

  • Results

    NeRCo performs against a rich set of state-of-the-art algorithms and even outperforms some supervised methods.

  • Takeaways & Limitations

    The method recovers visual-friendly low-light results in an unsupervised manner while reducing reliance on paired data.

Abstract

from arXiv · show

The following three factors restrict the application of existing low-light image enhancement methods: unpredictable brightness degradation and noise, inherent gap between metric-favorable and visual-friendly versions, and the limited paired training data. To address these limitations, we propose an implicit Neural Representation method for Cooperative low-light image enhancement, dubbed NeRCo. It robustly recovers perceptual-friendly results in an unsupervised manner. Concretely, NeRCo unifies the diverse degradation factors of real-world scenes with a controllable fitting function, leading to better robustness. In addition, for the output results, we introduce semantic-orientated supervision with priors from the pre-trained vision-language model. Instead of merely following reference images, it encourages results to meet subjective expectations, finding more visual-friendly solutions. Further, to ease the reliance on paired data and reduce solution space, we develop a dual-closed-loop constrained enhancement module. It is trained cooperatively with other affiliated modules in a self-supervised manner. Finally, extensive experiments demonstrate the robustness and superior effectiveness of our proposed NeRCo. Our code is available at https://github.com/Ysz2022/NeRCo.

1. Introduction

Low-light enhancement is important because brightness degradation harms image content and downstream vision tasks, while limited supervision and visually poor references constrain existing methods. NeRCo addresses these issues through controllable neural representation, multimodal supervision, and unsupervised cooperative training.

  • Brightness degradation obscures objects and reduces contrast, impairing downstream tasks such as semantic segmentation and object detection.
  • Existing methods face unpredictable low-light factors, limited supervised datasets, and a gap between metric-oriented and visual-friendly results.On LIME, recently proposed top-performing algorithms are reported to exhibit severe color cast.
  • Neural representation normalizes unpredictable degradation by fitting the degraded scene controllably and selectively avoiding extreme degradation during enhancement.
  • Multimodal supervision combines image and textual guidance so outputs can match references while also aligning with more stable brightness descriptions.
  • Unsupervised cooperative adversarial learning reduces reliance on paired training data while appearance-based discrimination preserves authenticity at color and detail levels.
  • NeRCo performs against a rich set of state-of-the-art algorithms and even outperforms some supervised methods.

2. Related Work

Prior low-light enhancement work spans model-based, deep learning, neural representation, and vision-language approaches. NeRCo extends neural representation and pre-trained vision-language priors to low-light enhancement.

  • Model-based methods estimate physical degradation parameters but struggle with diverse low-light factors and require extensive hand-crafted priors.
  • Deep learning methods recover images data-dependently, but prior work includes persistent challenges such as heavy noise and reliance on specialized constraints.
  • Neural representation has mainly been used for compression, denoising, continuous image depiction, super-resolution, warping, and shape-related applications.
  • NeRCo is presented as the first method to apply neural representation’s controllable fitting capability to low-light image enhancement.
  • Pre-trained vision-language models such as CLIP provide transferable semantic priors, previously used mainly for high-level computer vision tasks.
  • NeRCo applies vision-language priors to low-light enhancement through semantic-oriented guidance for improved performance.

3. Our Method

NeRCo combines neural-representation normalization, multimodal appearance supervision, and dual-loop cooperative training for low-light enhancement. These components normalize degradation, guide perceptually consistent outputs, and constrain bidirectional mappings without paired references.

  • Overall Framework: NeRCo first normalizes low-light inputs with a neural representation module before enhancement, improving robustness to varying degradation conditions.The module represents images through feature maps and pixel coordinates, then decodes predicted RGB values.
  • 3.2. Neural Representation for Normalization: NRN controls fitting capacity through positional encoding, allowing degradation normalization while avoiding excessive reproduction of unwanted degradation components.Increasing L produces a more precise fit, but the paper argues that stronger fitting is not always preferable for normalization.
  • 3.3. Text-driven Appearance Discriminator: TAD uses CLIP-derived text and image modalities to supervise semantic consistency, while appearance discrimination supplies image-level guidance for color, texture, and structure.The text branch compares image and prompt embeddings with cosine similarity; the appearance branch also includes a high-frequency path.
  • 3.4. Dual Loop Generation Procedure: The dual-loop procedure alternates enhancement and degradation mappings, converting IL to eIH and back to eeIL, while also supporting the reverse high-light-to-low-light branch.GH denotes enhancement and GL denotes degeneration; inference directly enhances low-light inputs, whereas the high-light branch is used for training.
  • 3.4. Dual Loop Generation Procedure: Cycle consistency constrains the bidirectional mappings at the pixel level, and Cooperative Loss makes the modules mutually supervise attention guidance and reduces the solution space.For low-light inputs, extracted and estimated lightness masks and reconstructed images are encouraged to agree.

4. Experiments

Experiments evaluate NeRCo across paired and unpaired low-light benchmarks, compare it with model-based, supervised, and unsupervised methods, and analyze its components through quantitative and visual ablations.

  • Benchmarks and Metrics: NeRCo is evaluated on LSRW, LOL, and LIME, including a setting trained only with low-light images and BSD300 replacements for paired references.LSRW contains 1,000 training pairs and 50 evaluation pairs; LOL and LIME provide additional evaluation settings.
  • Comparison with the State-of-the-Art: The comparison includes two model-based, three supervised, and five unsupervised low-light enhancement methods.The baselines include LECARM, SDD, RetinexNet, KinD, URetinex-Net, ZeroDCE, SSIENet, RUAS, EnGAN, and SCI.
  • Quantitative Analysis: NeRCo achieves nearly SOTA performance on both full-reference and no-reference metrics across all benchmarks, including performance better than supervised methods.The authors attribute advantages over EnGAN and SCI to stronger visual-oriented constraints and normalization robustness.
  • Qualitative Analysis: On severely degraded images, NeRCo produces prominent contrast, vivid colors, and preserved details, while other methods show insufficient brightness, oversmoothing, veils, or artifacts.The qualitative comparison identifies NeRCo as having the best visual quality in the reported scenario.
  • Ablation Study: The ablation study successively adds mask extraction, cooperative loss, edge-path discrimination, text-driven appearance discrimination, and neural representation normalization to the dual loop.The four settings are evaluated on LSRW, with the complete model forming the final configuration.
  • Ablation Study: The complete NeRCo setting achieves apparent performance gains and the best perceptual effectiveness, especially in the regions boxed in green and red.The reported explanation links the result to neural representation normalization and text-driven appearance discrimination.

5. Conclusion

NeRCo is an implicit neural representation method for unsupervised low-light enhancement that combines degradation normalization, perceptual supervision, and cooperative dual-loop constraints. Experiments report superiority over other top-performing methods, while the components are suggested as inspiration for other low-level vision tasks.

  • Conclusion: NeRCo normalizes dark lightness and natural noise with neural representation, uses high-frequency and vision-language supervision, and trains enhancement through dual-closed-loop cooperative constraints.The constraints are intended to reduce reliance on paired data and solution space.
  • Conclusion: Experiments report that NeRCo is superior to other top-performing low-light enhancement methods.
  • Conclusion: The proposed components provide inspiration for image dehazing, compressive sensing, and hyperspectral imaging.

Abstract

The supplementary material extends NeRCo's evaluation through normalization, prompt, discriminator-path, semantic, and qualitative analyses. It reports stable prompt-based results, best textual-description accuracy, and best performance across three benchmarks.

  • Supplementary Analyses: The supplementary material analyzes neural representation normalization, prompt choices, text-driven supervision, discriminator paths, semantic classification, and additional qualitative results.
  • Semantic Analysis: Enhanced results from NeRCo achieve the best accuracy for textual descriptions of high-light images using a pre-trained vision-language model.
  • Qualitative Analysis: Additional qualitative analyses on LSRW, LOL, and LIME report NeRCo as achieving the best performance.

A. Normalized Results

The normalized-results analysis examines whether NRN reduces variation in brightness across differently exposed images with the same content. It also studies text-driven supervision under alternative prompts without the neural representation function.

  • Normalized Results: NRN processes image sets with different brightness and identical content, producing outputs with similar brightness.The examples are drawn from three SICE image sets and matched by colored boxes.
  • Prompt Analysis: The prompt experiment uses dark/bright, dim/light, and night/day pairs on the #3 ablation setting, which removes neural representation.
  • Prompt Analysis: All tested prompt variants achieve decent scores across four metrics, with values in a stable range above other ablations and below NeRCo.
  • Prompt Analysis: The prompt study reports a performance gain from text-driven supervision.

C. TAD Ablation

The TAD ablation evaluates color, edge, and text-driven supervision, showing that edge and text paths improve results. Prompt-based settings also outperform versions without prompts across the reported study.

  • TAD components: Edge supervision and text supervision each improve TAD’s effect over the color-discrimination base.Color discrimination is retained as the base because at least one supervision path is required.
  • Prompt options: Prompt-trained settings outperform versions without prompts, and synonym prompts can replace the original prompts.
  • Experimental setup: The study reports TAD ablation results on the LSRW dataset using ablation setting “#3” as the reference.

D. Semantic Evaluation

Semantic evaluation uses CLIP-based image–text consistency scores to assess whether enhanced outputs match a high-light-image prompt. NeRCo achieves the best reported semantic scores, including scores above the ground truth in the described comparison.

  • Evaluation method: CLIP generates image and text vectors, whose cosine discrepancy is converted by softmax into a semantic score from 0 to 1.
  • Results: NeRCo achieves the best semantic scores among the compared methods and scores higher than the ground truth in the reported evaluation.The low-light input receives the lowest likelihood of being a high-light image, while high-light references are generally classified accurately except on LIME.

E. Qualitative Analysis

Qualitative comparisons across LSRW, LOL, and LIME show NeRCo producing visually friendly outputs with authentic tones. The analysis emphasizes simultaneous color adjustment and detail preservation relative to competing methods.

  • Comparison scope: The qualitative analysis supplements the paper’s quantitative results with additional comparisons against multiple state-of-the-art enhancement methods.
  • LSRW: On LSRW, model-based methods leave images insufficiently bright, while several other methods introduce color casts or discard details during denoising.NeRCo is described as preserving details while adjusting color.
  • Cross-dataset comparison: Across LOL, LIME, and LSRW comparisons, NeRCo is reported to recover the most authentic tones and visually friendly results.
Loading 2303.11722v3…