Source-linked AI summary
A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
Patryk Chrabaszcz, Ilya Loshchilov, Frank Hutter
TL;DR
The computational cost of experimenting on full ImageNet motivates cheaper benchmarks that retain its complexity. The paper provides downsampled ImageNet variants and finds strong results, similar qualitative behavior across resolutions, and potentially much cheaper experimentation.
Problem
Full ImageNet experiments impose a large computational burden, motivating cheaper alternatives that preserve dataset complexity and support architecture and hyperparameter evaluation.
Method
The paper provides ImageNet variants downsampled to 64×64, 32×32, and 16×16 pixels while retaining ImageNet’s number of classes and images, then analyzes hyperparameters and network sizes.
Results
Downsampled ImageNet yields surprisingly strong classification results, with qualitative findings similar across resolutions and Wide ResNets on ImageNet32x32 matching AlexNet’s full-sized ImageNet baseline in reported top-5 error.
Takeaways & Limitations
The datasets may fill the gap between CIFAR and full ImageNet benchmarks, supporting faster experimentation and potentially transferring findings from smaller, lower-resolution settings to larger ones.
Abstract
from arXiv · showhide
The original ImageNet dataset is a popular large-scale benchmark for training Deep Neural Networks. Since the cost of performing experiments (e.g, algorithm design, architecture search, and hyperparameter tuning) on the original dataset might be prohibitive, we propose to consider a downsampled version of ImageNet. In contrast to the CIFAR datasets and earlier downsampled versions of ImageNet, our proposed ImageNet32$\times$32 (and its variants ImageNet64$\times$64 and ImageNet16$\times$16) contains exactly the same number of classes and images as ImageNet, with the only difference that the images are downsampled to 32$\times$32 pixels per image (64$\times$64 and 16$\times$16 pixels for the variants, respectively). Experiments on these downsampled variants are dramatically faster than on the original ImageNet and the characteristics of the downsampled datasets with respect to optimal hyperparameters appear to remain similar. The proposed datasets and scripts to reproduce our results are available at http://image-net.org/download-images and https://github.com/PatrykChrabaszcz/Imagenet32_Scripts
1 INTRODUCTION
ImageNet offers more complex benchmarking than CIFAR but requires substantial computation, motivating cheaper alternatives that preserve its complexity. The paper proposes downsampled ImageNet variants and reports strong results with similar qualitative behavior across resolutions.
- Training strong models on ImageNet typically requires several GPU months, creating a substantial computational burden.
- Downscaled ImageNet is proposed to make experimentation with neural architectures and hyperparameters cheaper while preserving dataset complexity.
- Lower-resolution images could make classification more difficult and postpone saturation observed on CIFAR-10 benchmarks.
- The proposed variants yield surprisingly strong classification results, with qualitative results remaining similar across downsampling sizes.
- Wide ResNets on ImageNet32x32 match AlexNet’s 18.2% top-5 error while using images with roughly 50× fewer pixels per image.
2 DOWNSAMPLING IMAGENET
The paper constructs labeled, low-resolution ImageNet datasets at three resolutions while retaining ImageNet’s full class and image counts. It evaluates multiple downsampling methods to assess their effect on classification quality.
- Original ImageNet contains 1000 object classes, variable training images per class, and 50 validation images per class.
- Earlier low-resolution alternatives differ in scope: Downsampled ImageNet lacks labels, TinyImageNet contains 200 classes, and a proposed 128×128 dataset was unavailable.
- The proposed ImageNet16x16, ImageNet32x32, and ImageNet64x64 datasets retain the original numbers of classes and images and provide labeled training and validation files.
- The datasets use a CIFAR-compatible file format, with shuffled data split across training files and a separate validation file.
- Six Pillow downsampling techniques are evaluated by training Wide Residual Networks and comparing validation errors.
3 EXPERIMENTAL SETUP
The experiments train Wide Residual Networks with varying widths under a scheduled stochastic-gradient-descent protocol. Architectures are adapted to each image resolution, and horizontal flipping doubles the images used per epoch.
- The study trains WRN-N-k models, where N controls depth and k controls network width, with k = 1 corresponding to 16 filters in the first residual block.
- Stochastic Gradient Descent uses momentum 0.9, reduces the learning rate by 5.0 every 10 epochs, and runs for up to 40 epochs.
- Validation error rates are reported after 31 epochs, immediately after the final learning-rate drop.
- WRNs are modified for 64×64 inputs by adding residual blocks and for 16×16 inputs by removing the last residual-block stack.
- Horizontal image flipping is concatenated with the original training images, effectively doubling the number of images per epoch.
4 RESULTS
The experiments show that downsampling choice, image resolution, network width, and learning rate shape ImageNet performance, while several qualitative conclusions remain stable across settings. Downsampled ImageNet supports strong accuracy and faster experimentation, especially when computational tradeoffs are considered jointly.
- Downsampling technique: Nearest-neighbor downsampling performed worst for every tested learning rate, while the other five techniques produced similar results.This pattern also held for ImageNet16x16 and ImageNet64x64; subsequent experiments used box downsampling.
- Network size and resolution: 12.64% Top-5 performance was obtained on ImageNet64x64, showing that greater image resolution yielded better results.Larger network width also improved results independently of downsampling size.
- Hyperparameter stability: Optimal learning-rate regions remained similar across downsampling sizes and network widths whose space and time complexity differed by up to 100×.Performance as a function of learning rate and width was qualitatively similar for ImageNet32x32 and ImageNet16x16.
- Training-time tradeoffs: Reducing computational cost through both downsampling and network-size reduction improved the performance-versus-training-time tradeoff.The paper suggests exploiting consistent learning-rate rankings across settings through cheap proxies for more expensive architecture and hyperparameter searches.
5 DISCUSSION AND CONCLUSION
The downsampled ImageNet variants may offer a viable alternative to CIFAR while retaining more complex data and classes. They could support substantially cheaper experimentation and potentially transfer findings from smaller, lower-resolution settings to larger ones.
- Downsampled ImageNet variants may serve as alternatives to CIFAR while retaining more complex data and classes.
- Low-resolution classification is relevant when storage matters, images are noisy, or only a small image region requires classification.The original ImageNet dataset is 145GB.
- The datasets are intended as benchmarks for algorithm design, architecture search, and hyperparameter optimization.
- Preliminary experiments suggest findings from smaller networks and lower-resolution images may transfer to larger networks and higher-resolution images at up to 100 times lower cost.The authors frame this as a hypothesis that could support multi-fidelity architecture and hyperparameter search.