PaperlayerView exact revision on arXiv

Source-linked AI summary

Multitask AET with Orthogonal Tangent Regularity for Dark Object Detection

Ziteng Cui, Guo-Jun Qi, Lin Gu, Shaodi You, Zenghui Zhang, Tatsuya Harada

arXiv:2205.03346v1cs.CV

TL;DR

Low-light conditions challenge object detection, and enhancement methods or existing training data do not fully address illumination and distribution differences. MAET learns degradation-aware representations from realistic sensor and ISP transformations, jointly decodes object detections, and separates the tasks with orthogonal tangent regularity. The authors report state-of-the-art performance on synthetic and real-world dark-image datasets.

  • Problem

    Existing methods face target and data inconsistencies, while human-oriented enhancement can produce artifacts that do not benefit high-level visual understanding.

  • Method

    MAET encodes normal-lit and low-light images, decodes physically modeled degradation parameters and object detections, and uses orthogonal tangent regularity to disentangle the tasks.

  • Results

    The proposed algorithm outperforms state-of-the-art models on real-world and synthetic dark-image datasets.

  • Takeaways & Limitations

    MAET provides a unified framework for transformation-equivariant representation learning and dark-image object detection using mainstream detection architectures.

  • Takeaways & Limitations

    The ISP model omits demosaicing and other steps to balance precision and generalizability, and the influence of demosaicing is analyzed separately.

Abstract

from arXiv · show

Dark environment becomes a challenge for computer vision algorithms owing to insufficient photons and undesirable noise. To enhance object detection in a dark environment, we propose a novel multitask auto encoding transformation (MAET) model which is able to explore the intrinsic pattern behind illumination translation. In a self-supervision manner, the MAET learns the intrinsic visual structure by encoding and decoding the realistic illumination-degrading transformation considering the physical noise model and image signal processing (ISP). Based on this representation, we achieve the object detection task by decoding the bounding box coordinates and classes. To avoid the over-entanglement of two tasks, our MAET disentangles the object and degrading features by imposing an orthogonal tangent regularity. This forms a parametric manifold along which multitask predictions can be geometrically formulated by maximizing the orthogonality between the tangents along the outputs of respective tasks. Our framework can be implemented based on the mainstream object detection architecture and directly trained end-to-end using normal target detection datasets, such as VOC and COCO. We have achieved the state-of-the-art performance using synthetic and real-world datasets. Code is available at https://github.com/cuiziteng/MAET.

1. Introduction

Low-light enhancement can introduce artifacts that hinder object detection, while existing methods separate human- and machine-vision targets and struggle with illumination variance. MAET unifies degradation modeling and object detection, using orthogonal tangent regularity to disentangle their outputs.

  • Enhancement methods optimized for human perception may generate artifacts that mislead high-level visual understanding tasks.
  • Existing approaches exhibit target inconsistency between human-vision and machine-vision objectives, while data inconsistency arises from mismatched training and evaluation illumination.
  • MAET learns illumination-degrading transformations from normal-lit and low-light image pairs using physical sensor noise and ISP modeling, then predicts object locations and categories.
  • Orthogonal tangent regularity geometrically disentangles object-detection outputs from transformation-decoding outputs along a shared parametric manifold.
  • The framework supports end-to-end training with standard detection datasets and can be applied to mainstream object detectors.
  • The authors report superior performance on low-light object detection compared with other methods.

2. Related Work

Related work addresses low-light vision through enhancement, augmented or synthetic data, task-specific robustness, and self-supervised representation learning. MAET extends auto-encoding transformations to learn transformation-equivariant features for dark-image object detection.

  • Existing low-light object-detection datasets cover pedestrians, faces, adverse conditions, poor visibility, and multi-class dark-object scenarios.
  • Low-light enhancement methods include Retinex-based, histogram-equalization, CNN-based, and GAN-based approaches focused on restoring details or correcting color shifts.
  • High-level dark-environment vision commonly uses enhancement post-processing, augmented real-world data, or oversimplified synthetic data.
  • MAET extends auto-encoding transformations, previously used for self-supervised tasks, to object detection in dark images.

3. Multitask Autoencoding Transformation (MAET)

MAET extends auto-encoding transformation learning to jointly model realistic low-light degradation and object detection. It uses ISP- and sensor-informed transformations, shared representation learning, and orthogonal tangent regularity to separate the two tasks.

  • 3.1. Background: From AET to MAET: AET learns latent features from an image and its transformed counterpart that decode the parameterized transformation.The transformation decoder estimates t from E(x) and E(t(x)), trained with transformation loss.
  • 3.2. Multi-Task AET with Orthogonal Regularity: MAET jointly decodes low-light degradation parameters and object locations and categories from encoded image representations.Its encoder is paired with degradation and object-detection decoders for multitask prediction.
  • 3.2. Multi-Task AET with Orthogonal Regularity: Orthogonal tangent regularity minimizes cosine similarity between degradation and detection tangents to geometrically disentangle their outputs.The tangents describe representation directions associated with decoder outputs, so orthogonality reduces cross-task influence.
  • 3.2. Multi-Task AET with Orthogonal Regularity: The total training objective combines orthogonal regularity, object-detection loss, and degradation-transformation loss.For YOLOv3, the detection loss includes localization, classification, and confidence terms, while degradation loss uses AET loss.
  • 3.3.1 Image Signal Processing (ISP) Pipeline: The degradation transform unprocesses sRGB images, applies low-light corruption, and reprocesses them through a simplified ISP pipeline.The pipeline models sensor and imaging operations including quantization, white balance, color-space conversion, gamma correction, and their inverses.

4. Experiments

Experiments evaluate MAET on synthetic low-light VOC and COCO data, then on real-world ExDark and UG2+ DARK FACE datasets using baseline, enhancement-based, and ablated comparisons. The reported results show improved detection performance for MAET across these settings.

  • Experimental setup: VOC evaluation reports mean average precision at an IoU threshold of 0.5, while COCO results include each evaluated COCO index.VOC uses VOC 2007 and 2012 training data with VOC 2007 test evaluation; COCO uses COCO 2017 train and validation sets.
  • Experimental setup: Experiments compare YOLO baselines, enhancement-preprocessed inputs, and MAET variants with and without orthogonal loss.The comparison includes synthetic low-light evaluation on VOC and COCO, plus real-world evaluations on ExDark and UG2+ DARK FACE.
  • Synthetic evaluation: MAET significantly improves the baseline detection framework on synthetic low-light data and outperforms enhancement methods across all reported evaluation indexes.The comparison covers MBLLEN, KIND, and Zero-DCE preprocessing methods.
  • Real-world evaluation: On ExDark, MAET achieves satisfactory performance for most object classes and overall mAP, supporting its evaluation in real-world dark conditions.ExDark contains 7,363 low-light images across 12 object categories, with category-wise AP and overall mAP reported.
  • Real-world evaluation: On UG2+ DARK FACE, MAET achieves better results than the other evaluated methods.The dataset contains 6,000 labeled low-light face images, with 5,400 used for fine-tuning and 600 for evaluation.

5. Conclusion

The paper concludes that MAET learns illumination-degradation-equivariant representations for dark-image detection and uses orthogonal tangent regularity to separate object and degradation features. It reports superior performance on synthetic and real-world dark-image datasets.

  • Conclusion: MAET learns an intrinsic representation equivariant to illumination degradation and decodes it for object detection in dark environments.The framework uses self-supervised degradation prediction alongside object detection.
  • Conclusion: Orthogonal tangent regularity geometrically disentangles object and degradation features by maximizing orthogonality between task-output tangents.The multitask predictions are modeled along a parametric manifold.
  • Conclusion: MAET outperforms state-of-the-art models on synthetic and real-world dark-image datasets.The conclusion summarizes results across both evaluation settings.

6. Display of Object Detection Results in Real-World Situation

Real-world examples from UG2+ DARK FACE and ExDark illustrate MAET’s object-detection performance under actual dark-light conditions. The figures compare MAET-YOLO with YOLO results using original or enhanced inputs.

  • Real-world examples: MAET shows better detection performance on actual dark-light detection examples from UG2+ DARK FACE and ExDark.The examples are presented in Figures 6 and 7.

7. Ablation Study

The ablation study compares low-light synthesis strategies and examines their impact on dark-object detection. The proposed synthesis method provides the greatest reported benefit on real-world datasets, while the paper also discusses simplified ISP processing and alternative synthesis baselines.

  • Synthesis baselines: Prior synthesis baselines use Retinex or inverse gamma correction, with inverse-gamma approaches optionally adding Poisson or Gaussian-Poisson noise.These methods are compared with the proposed degradation transformation.
  • ISP ablation: The simplified ISP procedure omits demosaicing as a tradeoff between processing precision and generalizability across images from varied sources.The influence of demosaicing is analyzed separately in supplementary material.
  • Data-generation comparison: The study compares Retinex, inverse-gamma, noisy inverse-gamma, and the proposed synthesis methods on ExDark and UG2+ DARK FACE.The proposed variants include versions with and without mosaicing and demosaicing.
  • Data-generation comparison: The proposed synthetic method provides the greatest help in improving detection performance on the real-world ExDark and UG2+ DARK FACE datasets.Training configurations and strategies are held consistent with the main experiments.
Loading 2205.03346v1…