Source-linked AI summary
KeepAugment: A Simple Information-Preserving Data Augmentation Approach
Chengyue Gong, Dilin Wang, Meng Li, Vikas Chandra, Qiang Liu
TL;DR
Data augmentation can introduce noisy examples and hurt performance when important image information is removed. KeepAugment uses saliency maps to preserve important regions during augmentation, improving prior augmentation methods across image classification, semi-supervised learning, tracking, and object detection.
Problem
Data augmentation may introduce noisy training examples and reduce performance when augmentation destroys information important for classification.
Method
KeepAugment scores image-region importance with saliency maps and preserves high-importance regions by avoiding their removal or pasting them back after transformation.
Results
KeepAugment significantly improves prior data augmentation baselines across image classification, semi-supervised learning, multi-view multi-camera tracking, and object detection.
Takeaways & Limitations
Information-preserving augmentation improves the fidelity of augmented training examples across the evaluated vision tasks.
Takeaways & Limitations
KeepAugment requires saliency-map computation via back-propagation at each training step, and naive implementation roughly doubles computational cost.
Abstract
from arXiv · showhide
Data augmentation (DA) is an essential technique for training state-of-the-art deep learning systems. In this paper, we empirically show data augmentation might introduce noisy augmented examples and consequently hurt the performance on unaugmented data during inference. To alleviate this issue, we propose a simple yet highly effective approach, dubbed \emph{KeepAugment}, to increase augmented images fidelity. The idea is first to use the saliency map to detect important regions on the original images and then preserve these informative regions during augmentation. This information-preserving strategy allows us to generate more faithful training examples. Empirically, we demonstrate our method significantly improves on a number of prior art data augmentation schemes, e.g. AutoAugment, Cutout, random erasing, achieving promising results on image classification, semi-supervised image classification, multi-view multi-camera tracking and object detection.
1. Introduction
Data augmentation can improve training diversity but may destroy classification-relevant information and introduce noisy or ambiguous examples. KeepAugment preserves important image regions during augmentation and improves prior augmentation methods across several vision tasks.
- 1. Introduction: Data augmentation can increase effective data size while degrading performance when it removes information needed for classification.Random Cutout and RandAugment may create examples with wrong or ambiguous labels by destroying key characteristics.
- 1. Introduction: KeepAugment preserves high-importance regions identified by saliency maps, either by avoiding their removal or pasting them back after transformation.The approach supports region-level methods such as Cutout and image-level methods such as RandAugment and AutoAugment.
- 1. Introduction: Across CIFAR-10, ImageNet, tracking, semi-supervised learning, and object detection, KeepAugment significantly improves prior data augmentation baselines.The paper reports 98.7% CIFAR-10 test accuracy with PyramidNet-ShakeDrop and improvements on COCO detection baselines.
- 1. Introduction: Figure 1 contrasts augmentation that damages important regions with KeepAugment variants that preserve them through selective cutting or patch pasting.The examples use ImageNet images and illustrate KeepCutout and KeepRandAugment.
2. Data Augmentation
The paper distinguishes region-level augmentation, which masks or modifies local image regions, from image-level augmentation, which transforms whole images. It motivates KeepAugment by showing that stronger augmentation can lose information and introduce noise when insufficiently controlled.
- 2. Data Augmentation: Label-invariant augmentation generates transformed images expected to retain the original label, using a stochastic transform A such that x′ = A(x).The discussion focuses on two widely used classes of computer-vision augmentation.
- 2. Data Augmentation: Region-level methods such as Cutout and random erasing create partially occluded examples by masking or modifying random rectangular image regions.These operations can be represented with randomly generated binary masks over the input image.
- 2. Data Augmentation: Image-level methods apply label-invariant transformations such as solarization, sharpness, posterization, and color normalization to whole images.AutoAugment searches transformation compositions with reinforcement learning, while later methods reduce the search burden.
- 2. Data Augmentation: Increasing augmentation strength can cause information loss, noise, and ambiguity when the transformation is not properly controlled.The paper studies this trade-off by varying Cutout length and RandAugment distortion magnitude on CIFAR-10.
3. Our Method
KeepAugment preserves informative image regions during augmentation by using saliency scores to guide selective cutting or pasting. The method includes threshold-based sampling and efficient saliency approximations that reduce computational overhead without reported accuracy loss.
- KeepAugment: KeepAugment uses saliency maps to preserve high-importance regions, avoiding them during Cutout or pasting them back after image-level transformations.The approach measures regional importance and applies augmentation adaptively to improve augmented-image fidelity.
- Selective-Cut: Selective-Cut repeatedly samples regions until their importance score falls below threshold τ, thereby restricting augmentation from masking critical areas.The augmented image uses a binary mask for the selected region.
- Selective-Paste: Selective-Paste augments the image first, then samples a region with importance above τ and pastes that original region onto the transformed image.This procedure preserves informative content while retaining the broader image-level transformation.
- Threshold selection: The adaptive threshold is chosen as a quantile of candidate-region importance scores, with selective-cut using below-threshold regions and selective-paste using above-threshold regions.This thresholding procedure is applied for a specified region size and sampled uniformly among qualifying regions.
- Threshold analysis: On CIFAR-10, selective-cut improves over standard Cutout at relatively small thresholds, whereas performance drops when increasingly important regions are cut.Figure 3 evaluates threshold effects for Cutout and RandAugment under fixed augmentation settings.
- Efficient implementation: Low-resolution saliency maps and an early auxiliary loss head reduce computation by about 3X without reported performance loss.The low-resolution strategy reduces ImageNet resolution from 224 to 112, while the early-head strategy estimates saliency from an early network layer.
4. Experiments
Across classification, semi-supervised learning, tracking, and detection, KeepAugment consistently improves prior augmentation baselines while preserving competitive training cost. Its strongest reported results include 98.7% CIFAR-10 accuracy, improved ImageNet top-1 accuracy, 95.0% Market1501 accuracy, and higher COCO detection mAP.
- CIFAR-10 Classification: 98.7% test accuracy on CIFAR-10 establishes a reported state-of-the-art result with PyramidNet-ShakeDrop without extra training data.
- CIFAR-10 Classification: Low-resolution or early-loss-head saliency computation makes training cost only slightly larger than Cutout while improving performance.
- CIFAR-10 Classification: KeepAugment consistently improves accuracy across CIFAR-10 architectures and augmentation baselines, including Cutout, CutMix, Random Erasing, and AutoAugment.
- ImageNet Classification: KeepAugment improves ImageNet top-1 accuracy over CutMix by 0.5% with ResNet-50 and 0.4% with ResNet-101, and over AutoAugment from 77.6% to 78.1% and 79.3% to 79.7%.
- Semi-supervised Image Classification: KeepAugment consistently improves UDA on CIFAR-10 across different numbers of labeled images in semi-supervised classification.
- Tracking and Object Detection: KeepAugment achieves 95.0% accuracy and 87.4 mAP on Market1501, while object detection improves from 37.9 to 39.1 mAP and from 39.9 to 41.2 mAP on RetinaNet.
5. Related Works
KeepAugment relates to approaches that balance augmentation fidelity and diversity, while selectively preserving informative regions without additional supervision. It is positioned as more general than methods tied to a single augmentation type or task.
- Comparison with prior methods: Attentive CutMix extracts informative regions but requires an additional pretrained classification model, whereas KeepAugment does not need additional supervision.
- Comparison with prior methods: KeepAugment preserves informative regions to improve fidelity while retaining diversified augmented examples, without requiring additional supervision.The method is contrasted with teacher-based approaches that correct noisy examples or extract informative regions.
- Comparison with prior methods: Teacher-based label correction also requires additional supervision and focuses on one typical augmentation method.
- Scope: The paper focuses on label-invariant augmentation and leaves further extensions to label-mixing methods as future work.The authors report empirical improvement on CutMix despite this stated focus.
- Saliency-guided augmentation: Saliency maps are used to measure regional importance for avoiding informative areas during cutting-based augmentation or improving image-level augmentation.
6. Conclusion
The paper presents saliency-guided adaptive augmentation as a way to reduce noisy training examples and improve prior data augmentation methods. Extensive evaluation reports gains across image classification, semi-supervised learning, tracking, and object detection.
- Conclusion: KeepAugment significantly improves prior data augmentation schemes across image classification, semi-supervised image classification, multi-view multi-camera tracking, and object detection.