Source-linked AI summary
Image Data Augmentation for Deep Learning: A Survey
Suorong Yang, Weikang Xiao, Mengchen Zhang, Suhan Guo, Jian Zhao, Furao Shen
TL;DR
Limited labeled image data motivate augmentation methods that increase training-data sufficiency and diversity. The paper proposes a taxonomy, systematically reviews methods across three computer-vision tasks, and experimentally compares them, while discussing unresolved theoretical, efficiency, and data-quality challenges.
Problem
Labeled image data may be limited, while existing augmentation research lacks broad task-oriented reviews, systematic comparisons, and theoretical guidance on sufficient dataset size.
Method
The paper proposes a taxonomy, reviews augmentation methods for semantic segmentation, image classification, and object detection, and evaluates methods and combinations on open image datasets.
Results
Data augmentation brings promising performance improvements for object detection and average accuracy improvement for image classification across the reported experiments.
Takeaways & Limitations
The survey presents data augmentation as an effective solution for shortages of limited labeled image data and identifies future research directions.
Takeaways & Limitations
The field lacks theoretical standards for sufficient training-data size, and augmentation can introduce noisy examples or negative inference effects.
Abstract
from arXiv · showhide
Deep learning has achieved remarkable results in many computer vision tasks. Deep neural networks typically rely on large amounts of training data to avoid overfitting. However, labeled data for real-world applications may be limited. By improving the quantity and diversity of training data, data augmentation has become an inevitable part of deep learning model training with image data. As an effective way to improve the sufficiency and diversity of training data, data augmentation has become a necessary part of successful application of deep learning models on image data. In this paper, we systematically review different image data augmentation methods. We propose a taxonomy of reviewed methods and present the strengths and limitations of these methods. We also conduct extensive experiments with various data augmentation methods on three typical computer vision tasks, including semantic segmentation, image classification and object detection. Finally, we discuss current challenges faced by data augmentation and future research directions to put forward some useful research guidance.
1 Introduction
Deep learning benefits from large, diverse datasets, but labeled image data remain limited and augmentation has received less attention than architectures and computing. This survey organizes augmentation methods, reviews them across computer-vision tasks, and evaluates them experimentally.
- Motivation: Deep learning progress depends on network architectures, computing power, and access to large datasets.Examples include deep ResNet architectures and datasets such as ImageNet, MS-COCO, and PASCAL VOC.
- Motivation: Data augmentation generates synthetic data to improve the sufficiency and diversity of training data.The augmented data are intended to represent a distribution close to the real one.
- Challenges: Image augmentation methods face challenges involving inefficiency, scalability, limited theory, and generating qualified data from very little data.The survey notes that dataset size is commonly chosen through personal experience and extensive experiments.
- Research gap: Prior reviews focused on narrower settings, including image classification, face recognition, or selected transformation categories.Related work also lacked systematic task coverage, effectiveness evaluations across actual tasks, or newer methods.
- Contribution: The paper proposes a taxonomy, reviews methods for object detection, semantic segmentation, and image classification, and compares methods and combinations experimentally.The experiments use various deep learning models and open image datasets.
2 Basic Data Augmentation Methods
Basic augmentation methods transform, erase, or mix image content directly. They are generally easy to implement, but their validity and effectiveness depend on data-distribution assumptions and task-specific constraints.
- Image Manipulation: Basic image manipulations apply direct transformations such as rotation, flipping, and cropping.These methods are presented as easy to implement.
- Image Manipulation: Basic manipulations assume existing data follow a distribution close to the actual data distribution.Translation and rotation can also lose boundary regions and require padding or interpolation.
- Image Erasing: Image erasing methods replace one or more image sub-regions with constant or random pixel values.Examples include Cutout, Hide-and-Seek, random erasing, and GridMask.
- Image Mix: Image mix methods combine two or more images or image sub-regions into one.Pairing samples averages pixel intensities, while Mixup combines samples and labels through convex combinations.
- Image Mix: CutMix replaces removed regions with patches from another image, producing more natural images than Mixup.Fmix instead uses random binary masks derived from thresholded low-frequency Fourier-space images.
3 Advanced Approaches
Advanced augmentation methods search policies automatically, transform learned features, or generate images with deep generative models. These approaches seek better coverage or efficiency but introduce search-cost, noise, data, and diversity considerations.
- Auto Augment: Auto-augmentation methods automatically search augmentation operations and magnitudes instead of relying only on manual design.Their search spaces contain policies specifying operations and application magnitudes.
- Auto Augment: Large policy search spaces and reinforcement-learning search algorithms create high time costs for auto-augmentation.Fast AutoAugment, PBA, and RandAugment reduce search cost through density matching, nonstationary schedules, or a smaller search space.
- Auto Augment: KeepAugment preserves informative regions detected by saliency maps, while Augmentation-wise Weight Sharing improves search efficiency during training.OHL-Auto-Aug jointly optimizes a parameterized augmentation distribution with network parameters.
- Feature Augmentation: Feature augmentation transforms vector representations in a learned feature space rather than only transforming input images.Methods include noise addition, nearest-neighbor interpolation or extrapolation, prototypical refinement, and moment exchange.
- Deep Generative Models: Deep generative models aim to generate samples from a distribution similar to the original data distribution.GANs are presented as representative methods, with Pix2Pix and CycleGAN used for image translation.
- Deep Generative Models: StarGAN v2 generates diverse images across multiple domains by separating domain categories from image-specific styles.This addresses the deterministic per-domain mapping described for StarGAN.
4 Evaluation
The paper evaluates image data augmentation across semantic segmentation, image classification, and object detection using common public datasets and models. Augmentation improves performance across all three tasks.
- 4 Evaluation: Evaluations cover semantic segmentation, image classification, and object detection using commonly used public datasets.The experiments use PASCAL VOC, CIFAR-10, CIFAR-100, SVHN, and COCO2017, with multiple representative models.
- 4.1 Semantic Segmentation: Semantic segmentation experiments apply manipulation-based augmentations to several models and report improvement using IoU and accuracy.Methods include flipping, scaling, rotation, noise injection, cropping, translation, and sharpening.
- 4.2 Image Classification: Image classification results compare augmentation and no augmentation across Wide-ResNet, DenseNet, and Shake ResNet.The datasets are CIFAR-10, CIFAR-100, and SVHN, using manipulation, erasing, and mixing methods.
- 4.2 Image Classification: Data augmentation leads to average accuracy improvement in the image classification experiments.Table 3 summarizes results on CIFAR-10, CIFAR-100, and SVHN.
- 4.3 Object Detection: Object detection experiments on COCO2017 compare FasterRCNN and CenterNet with and without augmentation using mAP, AP50, and AP75.The reported metrics are summarized across the evaluated augmentation methods.
5 Discussion for Future Directions
The discussion identifies unresolved theoretical, evaluation, data-volume, and augmentation-selection problems. It calls for stronger theory, unified evaluation, and better understanding of how augmentation affects data quality and performance.
- Theoretical Research on Data Augmentation: Data augmentation remains an auxiliary performance tool whose effectiveness is not fully explained theoretically.The paper highlights pairing samples and Mixup as methods that can improve accuracy despite limited understanding of why.
- Theoretical Research on Data Augmentation: There is no quantitative standard for sufficient training-dataset size, so dataset size is chosen through experience and extensive experiments.The suitable size depends on models, training objectives, and data-collection difficulty.
- The Evaluation of Data Augmentation Methods: Evaluating synthesized image quality remains open because no unified metrics are available.The paper links evaluation to the quantity and diversity of training data and model generalization ability.
- The Number of Generated Data: Increasing training-data quantity does not necessarily increase performance proportionally when additional data do not increase diversity.The paper identifies the appropriate amount of generated data as an unresolved research question.
- The Number of Generated Data: Oversampling minority classes can repeat existing examples, potentially saturating the minority class and causing overfitting.The discussion uses SMOTE as an example of minority-class oversampling.
6 Conclusion
The paper addresses limited labeled image data by comprehensively reviewing augmentation methods across computer vision tasks, organizing them into a taxonomy, and evaluating them empirically.
- Data augmentation is presented as an effective solution to shortages of limited labeled image data.
- The paper provides a comprehensive review of image data augmentation methods across various computer vision tasks.
- A taxonomy summarizes representative image data augmentation approaches in each category.
- The reviewed methods are compared empirically across various computer vision tasks.
- The paper discusses current challenges and highlights future research perspectives.