PaperlayerView exact revision on arXiv

Source-linked AI summary

Getting to Know Low-light Images with The Exclusively Dark Dataset

Yuen Peng Loh, Chee Seng Chan

arXiv:1805.11227v1cs.CV

TL;DR

Object detection lacks representative low-light data despite the importance of low-light vision and the prominence of bright-image benchmarks. The paper introduces the visible-light ExDARK dataset with image- and object-level annotations, then analyzes handcrafted and learned features to show that low-light alters object representations and attention beyond simple illumination invariance.

  • Problem

    Object detection lacks a go-to low-light benchmark, while public object datasets contain very little low-light data and low-light research often uses limited or nonrepresentative data.

  • Method

    The paper constructs ExDARK from visible-light low-light images with image- and object-level annotations, then analyzes handcrafted features, CNN representations, t-SNE embeddings, and activation maps.

  • Results

    Low-light alters object features and challenges feature attention; handcrafted features and denoising are insufficient, while increasing low-light training data improves low-light performance.

  • Takeaways & Limitations

    Low-light object detection requires dedicated data and careful feature analysis rather than relying on simple illumination invariance.

Abstract

from arXiv · show

Low-light is an inescapable element of our daily surroundings that greatly affects the efficiency of our vision. Research works on low-light has seen a steady growth, particularly in the field of image enhancement, but there is still a lack of a go-to database as benchmark. Besides, research fields that may assist us in low-light environments, such as object detection, has glossed over this aspect even though breakthroughs-after-breakthroughs had been achieved in recent years, most noticeably from the lack of low-light data (less than 2% of the total images) in successful public benchmark dataset such as PASCAL VOC, ImageNet, and Microsoft COCO. Thus, we propose the Exclusively Dark dataset to elevate this data drought, consisting exclusively of ten different types of low-light images (i.e. low, ambient, object, single, weak, strong, screen, window, shadow and twilight) captured in visible light only with image and object level annotations. Moreover, we share insightful findings in regards to the effects of low-light on the object detection task by analyzing visualizations of both hand-crafted and learned features. Most importantly, we found that the effects of low-light reaches far deeper into the features than can be solved by simple "illumination invariance'". It is our hope that this analysis and the Exclusively Dark dataset can encourage the growth in low-light domain researches on different fields. The Exclusively Dark dataset with its annotation is available at https://github.com/cs-chan/Exclusively-Dark-Image-Dataset

I. INTRODUCTION

Low-light vision remains underrepresented in object detection because existing research and datasets focus on enhancement, surveillance, or bright imagery. The paper introduces ExDARK and analyzes how low-light conditions affect object-focused vision.

  • Motivation: Low-light conditions affect everyday visibility, yet object detection research has largely focused on bright images.Existing low-light work commonly addresses image enhancement or surveillance using costly hardware rather than relatable assistive systems.
  • Contributions: The paper presents ExDARK as a large visible-light low-light collection with object-level annotation and an object-focused analysis using handcrafted and learned features.The analysis compares low-light vision with vision under sufficient illumination.
  • Related low-light data: Existing low-light enhancement datasets are inconsistent, often synthetically darkened, small, combined from multiple sources, or assembled ad hoc.Some enhancement datasets contain fewer than 100 images, while others use qualitative collections without a common standard.
  • Related low-light data: See-in-the-Dark uses raw short- and long-exposure images from selected cameras for enhancement, whereas ExDARK targets visible-light sRGB object data across broader low-light conditions.The source passage distinguishes raw-image processing from enhancement of sRGB images.

B. Object Datasets

PASCAL VOC, ImageNet, and Microsoft COCO established large-scale object benchmarks through increasing data, challenge, and annotation richness. However, their low-light coverage is limited, motivating ExDARK’s dedicated collection.

  • PASCAL VOC: PASCAL VOC grew through annual challenges, increasingly difficult images, additional annotations, and broader object coverage.It began with 4 object classes and 3,787 images sourced from existing datasets.
  • ImageNet: ImageNet provided over 1 million images and 1,000 image-level object classes, enabling CNN optimization and a new classification benchmark.Its images were challenging because an object could be labeled whenever an instance appeared.
  • Microsoft COCO: Microsoft COCO offered more than 300 thousand images and 80 object classes with annotations spanning detection, segmentation, and captions.Its advantage was the completeness of its image annotations rather than maximum image quantity.

III. EXCLUSIVELY DARK DATASET

The ExDARK dataset addresses the shortage of standardized low-light object data by collecting 7,363 images across diverse illumination conditions, with image- and object-level annotations. Its categorization exposes variation in low-light settings and supports object-focused analysis.

  • Motivation: The dataset was motivated by the absence of a go-to low-light benchmark and the limited low-light coverage of large public object datasets.Prior low-light enhancement datasets commonly used real images collected on an ad hoc basis.
  • Dataset overview: ExDARK contains 7,363 low-light images spanning 12 object classes, with image-level class labels and bounding boxes for every annotated instance.The classes include Bicycle, Boat, Bottle, Bus, Car, Cat, Chair, Cup, Dog, Motorbike, People, and Table.
  • Data collection: Images were collected from websites, existing datasets, movie frames, and manually captured photographs using different phones and cameras.Searches used keywords such as dark, low-light, and nighttime, while sampling included PASCAL VOC, ImageNet, and Microsoft COCO.
  • Dataset statistics: Most images contain one object instance, although some contain multiple instances, with up to 58 bounding boxes in one image.Image-level classes are relatively balanced, while People contributes most bounding-box annotations.
  • Low-light types: Ten illumination types organize the images across indoor and outdoor conditions, including low, ambient, object, single, weak, strong, screen, window, shadow, and twilight.The categories represent differences in illumination strength, visible light sources, location, and time of day.

IV. ANALYZING FEATURES IN LOW-LIGHT

The analysis evaluates how commonly used object features behave in low-light images by comparing ExDARK with a brightness-matched subset of Microsoft COCO. It considers both hand-crafted proposal features and learned CNN features.

  • Feature analysis: The evaluation covers hand-crafted object-proposal features and learned object-classification CNN features to examine low-light effects across feature types.The analysis compares behavior in low-light and bright images rather than evaluating only one feature family.
  • Experimental setup: The study compares feature behavior on ExDARK and bright images from a randomly sampled Microsoft COCO subset matched to the 12 object classes.Only annotations for those 12 classes are retained in the COCO analysis.

A. Performance of Hand-Crafted Features

The hand-crafted-feature analysis tests object proposal methods that rely on edges, gradients, or superpixel contrast. These methods provide distinct proposal mechanisms for examining object localization in low-light images.

  • Rationale: The analysis uses these hand-crafted methods because object proposal systems still employ them for their speed and low computational complexity.The study examines whether classically designed features handle low-light images effectively.
  • Proposal mechanisms: Edge Boxes proposes regions by grouping edges and scoring the edges contained within each bounding box for objectness.Its proposal score reflects the likelihood that a box contains an object.
  • Proposal mechanisms: BING uses correlations between object boundaries and image-gradient norms, followed by SVM-based scoring and calibration.The method applies one SVM to binarized norm gradients and another to calibrate the final objectness score.
  • Proposal mechanisms: Adobe Boxes represents object parts with high-contrast superpixel groups called adobes and scores boxes using their spatial concentration.Adobe Boxes can also refine proposals from other methods, including BING in the AdobeBING combination.

1) Quantitative Evaluation:

The quantitative evaluation compares proposal quality on low-light ExDARK and bright COCO images using detection rate and recall. Low-light proposals show a stricter localization challenge and lower usefulness despite some higher detection rates at low IoU.

  • Evaluation protocol: The evaluation compares Edge Boxes, BING, Adobe Boxes, and AdobeBING using up to 1,000 proposals per image and varying IoU thresholds.Detection rate is detections divided by ground truths, while recall is detections divided by proposals.
  • Detection rate: At lower IoU, ExDARK has higher detection rates, but this relationship reverses as IoU increases.Because higher IoU imposes stricter localization criteria, the reversal indicates greater difficulty achieving accurate localization in low-light images.
  • Detection rate: Average detections are lower in ExDARK than COCO for every tested proposal method, despite the higher low-IoU detection rate.The authors attribute the low-IoU rate difference to COCO containing more ground-truth objects that remain undetected.
  • Recall: Recall is lower on ExDARK than on COCO for every proposal method, indicating that many low-light proposals are not valuable.This occurs even when some methods generate fewer average proposals per image on ExDARK.

2) Qualitative Evaluation:

Qualitative evaluation shows that proposal failures differ between bright COCO images and ExDARK: COCO failures are linked to small object scale, whereas ExDARK failures arise from low-light noise, blending, and unclear boundaries.

  • COCO images: COCO proposal failures, particularly for Edge Boxes and BING, occur when objects are too small for edge and gradient features to capture.Adobe Boxes and AdobeBING perform better because superpixels segment objects from the background more precisely, although they do not solve the problem completely.
  • ExDARK images: ExDARK proposal failures are attributed to low-light factors rather than object scale.Additional noise interferes with features, while object-background or object-object blending produces poor alignment with ground truth.
  • ExDARK images: Unclear gradient boundaries and similar low-valued object and background pixels prevent methods from distinguishing objects in difficult low-light conditions.These conditions particularly limit both gradient-based and superpixel-based proposals.
  • Low-light types: Edge Boxes has the best detection rate across all ten low-light types, while Ambient and Single have the best detection rates and Low and Strong the weakest.For recall, Object performs best and Low performs worst.
  • Low-light types: Detection performance is stronger when objects are near a light source, but very low light causes object-background blending and strong lighting introduces clutter and irregular illumination.Across hand-crafted approaches, detection rates remain below 70% for every low-light type.

4) The Noise Problem:

The analysis identifies noise as a significant low-light factor and tests whether BM3D denoising improves object-proposal features. Denoising helps edges only marginally and can degrade other proposal methods, indicating that missing signal also limits performance.

  • Noise analysis: Low-light image noise is treated as a notable component of feature degradation, motivating experiments on denoised ExDARK data.The experiments use BM3D because it performs well and is commonly used in low-light enhancement post-processing.
  • Denoising results: BM3D produces only a minor improvement for Edge Boxes and degrades BING, Adobe Boxes, and AdobeBING.The comparison evaluates proposal methods on ExDARK before and after denoising.
  • Denoising results: Denoising improves some edge features but can simultaneously increase artifacts because BM3D block filtering is not designed specifically for low-light conditions.The observed benefit is therefore feature-dependent rather than uniform across proposal methods.
  • Implications: The small detection-rate improvement after denoising indicates that low-light difficulty involves both noise and a lack of informative signals.The paper identifies denoising and low-light enhancement as potential research directions for these two aspects.

B. Insights from Learned Features

The learned-feature analysis fine-tunes ResNet-50 on varying mixtures of bright COCO and low-light ExDARK images, then examines classification behavior and feature representations. Results show that low-light performance depends on low-light training data and may require separate modeling from bright images.

  • Method: The study fine-tunes a pretrained ResNet-50 on COCO and ExDARK using different bright-to-low-light training ratios.The final classification layer is adapted from 1,000 ImageNet classes to 12 dataset object classes.
  • Classification performance: Models fine-tuned with less low-light data are weaker on low-light classification, with performance increasing as the low-light training ratio rises.The experiments maintain the same overall training-image count while varying the bright and dark proportions.
  • Classification performance: Balanced bright and low-light training does not produce the best overall classification performance; Model 6 performs best overall but not better than models specialized for bright or low-light images.This result is reported for the same classification task across COCO and ExDARK test data.
  • Implications: The authors infer that dataset-dependent performance supports a low-light-only dataset and that bright and low-light data may require separate modeling.The paper presents this as a deduction from the classification results rather than as a universal modeling rule.

2) Feature Analysis with t-SNE:

t-SNE visualizations show that learned representations organize images by object class but also clearly separate bright COCO from low-light ExDARK images. Within low-light data, representations retain environmental and lighting-condition structure, with weaker class clustering in some low-light regions.

  • Class structure: t-SNE embeddings of ResNet-50 features show recognizable grouping by object class, including close grouping of relatively similar Cat and Dog classes.The embeddings also contain considerable class confusion.
  • Bright versus low-light: Bright COCO and low-light ExDARK images form clearly separated clusters in the learned feature embedding.This separation appears even though the features are high-level representations used for object classification.
  • Bright versus low-light: The Cat and Dog regions show a distinct bright–low-light split, while low-light regions without clear class clusters indicate less robust learned features.The paper interprets these patterns as evidence that representations differ inherently between bright and low-light images.
  • Low-light conditions: Low-light embeddings show some indoor–outdoor separation and distinguish several lighting types, although Strong and Twilight images partly group together.Indoor images tend toward the upper half of the embedding, while outdoor images are more scattered.
  • Implications: The authors conclude that the CNN can unwittingly learn low-light properties that hinder object classification.This conclusion follows from the separation of lighting-related groups within learned representations.

3) Attention Analysis with Activation Maps:

The activation-map analysis visualizes where a trained classifier attends in low-light images. It finds that attention is often drawn to bright light sources, alongside or directly onto the object of interest.

  • Activation-map construction: Activation maps are extracted before the final pooling layer to preserve the spatial locations of model activations.The analysis uses Model 6's last convolution output before the fully connected layer.
  • Activation-map construction: The model's 7 × 7 × 2048 activation maps are aggregated by taking the maximum value at each spatial location.The resulting map is resized to the original image dimensions and superimposed on the image.
  • Observed attention: In low-light images, model attention is often drawn partially or entirely to bright sources of light.Correctly classified examples show light sources either inside the attention region or shining directly on the object.

V. SUMMARY AND CONCLUSION

The paper introduces ExDARK as a low-light object-focused dataset and analyzes how low-light conditions affect handcrafted and learned features. The findings indicate that low-light creates feature and attention challenges requiring more low-light data and dedicated study.

  • Contributions: ExDARK contains exclusively visible-light low-light images with image- and object-level annotations for up to 12 classes and up to 10 low-light conditions.It is intended as a go-to database for low-light research and application-focused investigation.
  • Feature analysis: Handcrafted features designed mainly for bright conditions cannot adequately address noise and missing detail common in low-light images.A state-of-the-art denoising algorithm is also insufficient for noise that frequently accompanies low-light data.
  • Feature analysis: Increasing the amount of low-light training data should improve performance in low-light conditions.This finding comes from training CNNs with both bright and low-light data.
  • Feature analysis: Low-light alters object features, so the same object produces substantially different features under bright and low-light conditions.Irregular illumination also challenges feature attention in ways not found in bright environments.
  • Scope and outlook: The study is focused on object-detection-based feature analysis, leaving further low-light-domain phenomena to be explored.The authors expect ExDARK to support future work on vision behavior and practical low-light tasks.
Loading 1805.11227v1…