Source-linked AI summary
GridMask Data Augmentation
Pengguang Chen, Shu Liu, Hengshuang Zhao, Xingquan Wang, Jiaya Jia
TL;DR
CNN training needs effective data augmentation, while existing information-dropping methods can delete too much or preserve continuous regions. GridMask uses structured, uniformly distributed deletion and improves results across image classification, object detection, and semantic segmentation, while remaining simple and efficient.
Problem
Information-dropping augmentation methods need to balance deletion and reservation because excessive deletion or preservation can undermine useful training information.
Method
GridMask removes spatially uniformly distributed, disconnected image regions, controlling deletion density and size as a simple information-dropping policy.
Results
GridMask improves baselines across ImageNet classification, COCO2017 detection, and Cityscapes segmentation, outperforming prior information-dropping methods and AutoAugment.
Takeaways & Limitations
GridMask is a simple, general, efficient augmentation policy that can serve as a baseline policy for future augmentation-search algorithms.
Takeaways & Limitations
The paper presents GridMask as one successful structured information-dropping method and leaves open whether other effective structures exist.
Abstract
from arXiv · showhide
We propose a novel data augmentation method `GridMask' in this paper. It utilizes information removal to achieve state-of-the-art results in a variety of computer vision tasks. We analyze the requirement of information dropping. Then we show limitation of existing information dropping algorithms and propose our structured method, which is simple and yet very effective. It is based on the deletion of regions of the input image. Our extensive experiments show that our method outperforms the latest AutoAugment, which is way more computationally expensive due to the use of reinforcement learning to find the best policies. On the ImageNet dataset for recognition, COCO2017 object detection, and on Cityscapes dataset for semantic segmentation, our method all notably improves performance over baselines. The extensive experiments manifest the effectiveness and generality of the new method.
1. Introduction
GridMask addresses the need for data augmentation in CNN training by structuring information deletion to balance removed and retained image regions. It reports improvements over existing strategies across classification, detection, and segmentation tasks.
- Motivation: CNNs require substantial training data because their millions of parameters can otherwise cause serious over-fitting.
- Motivation: Information dropping must avoid both excessive deletion, which can remove objects and context, and excessive preservation, which can leave objects untouched.
- Limitations of prior methods: Cutout and random erasing delete one continuous region, while Hide-and-Seek randomly deletes small squares, leaving chances of continuous deletion or preservation.
- GridMask: GridMask deletes spatially uniformly distributed squares, controlling their density and size to improve the balance between deletion and reservation.
- Results: 1.4% improves ImageNet baseline accuracy, from 76.5% to 77.9% with ResNet50, exceeding Cutout, HaS, and AutoAugment results.
- Results: GridMask improves Faster-RCNN-50-FPN COCO2017 mAP from 37.4% to 39.2% and Cityscapes PSPNet50 mIoU from 77.3% to 78.1%.
2. Releated Work
The paper situates GridMask among regularization and data-augmentation methods, presenting it as an information-dropping policy that can serve as a new baseline. Data augmentation modifies input data without changing network structure and applies broadly across tasks.
- Regularization: Regularization methods add noise or combine image information to reduce over-fitting and improve model generalization.
- Data augmentation: Data augmentation operates on input data rather than network structure and is easy to apply across many tasks.
- GridMask: GridMask belongs to information-dropping augmentation and is proposed as a new baseline policy for data augmentation.
3. GridMask
GridMask applies a binary grid-shaped mask to images, deleting spatially distributed square regions while controlling their density, size, position, and keep ratio. The design targets a better balance between information deletion and reservation, and experiments report fewer failure cases than prior approaches.
- GridMask: Each mask is formed by tiling units described by r, d, δx, and δy.r specifies the shorter gray-edge ratio, d the unit length, and δx and δy the distances from the first intact unit to the image boundaries.
- Parameter choices: The keep ratio k controls the balance between reserved and removed image regions; excessive preservation risks over-fitting, whereas excessive deletion risks under-fitting.
- Parameter choices: Increasing d enlarges each dropped square while keeping the keep ratio constant, and randomizing d increases image variety.
- Parameter choices: Smaller d can avoid most failure cases, but removing very small regions may be ineffective for convolutional operations.
- Failure cases: GridMask has a lower statistically measured failure probability than Cutout and HaS as removal-square sizes vary.The experiment uses 224 × 224 images, object sizes within [40, 160], and a keep ratio of 0.75.
- Training scheme: GridMask can be applied with a constant probability or with probability increased linearly across training epochs, with the latter empirically better for most experiments.
4. Experiments
Experiments evaluate GridMask across image classification, object detection, and semantic segmentation, including comparisons with established augmentation methods and ablations of its structure and parameters. GridMask consistently improves baselines, surpasses competing information-dropping approaches, and remains effective across models and training settings.
- ImageNet: GridMask raises ResNet50, ResNet101, and ResNet152 accuracy by 1.4%, 1.1%, and 1.4%, respectively, without careful hand tuning.The reported accuracies increase to 77.9%, 79.1%, and 79.7%.
- CIFAR10: GridMask improves CIFAR10 accuracy for ResNet18 from 95.28% to 96.54% and also improves WideResNet28-10 and ShakeShake-26-32.The reported gains are +1.26%, +1.11%, and +0.88%, respectively; combining GridMask with AutoAugment achieves SOTA results on these models.
- Ablation Study: Structured dropping outperforms random block dropping, while varying d changes accuracy and can increase network robustness.The reported accuracy values for different d range from 77.26 to 77.89, and random dropping decreases final accuracy.
5. Discussion and Conclusion
GridMask is presented as a simple, general, effective information-dropping policy that improves performance across recognition, detection, and segmentation. The authors also frame it as one successful structured design among potentially many.
- 5. Discussion and Conclusion: 1.4% higher than baseline on ImageNet recognition, 1.8% on COCO2017 object detection, and 0.9% on Cityscapes semantic segmentation.The reported improvements span three computer vision tasks.
- 5. Discussion and Conclusion: GridMask deletes uniformly distributed image areas to form a grid-shaped information-dropping pattern.The authors report this structured deletion as more effective than complete random location.
- 5. Discussion and Conclusion: The method’s effect is consistently stronger than other information-deletion-based data augmentation methods.The authors further propose GridMask as a baseline policy for future data augmentation search algorithms.
- 5. Discussion and Conclusion: GridMask is one successful structured information-dropping approach, and other effective structures may also exist.The authors hope this perspective motivates further study of effective structures and may improve reinforcement-learning-based search.