Source-linked AI summary
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Han Xiao, Kashif Rasul, Roland Vollgraf
TL;DR
MNIST’s accessibility makes it useful for rapid algorithm prototyping, but its simple digit task motivates a similarly accessible, more challenging benchmark. This paper introduces Fashion-MNIST, a drop-in replacement built from standardized fashion-product images, with reported benchmark accuracies including 0.880 on Fashion-MNIST and 0.969 on MNIST for GradientBoostingClassifier.
Problem
MNIST supports rapid prototyping through its accessibility, while its simple digit task motivates a more challenging benchmark with the same usability.
Method
The paper constructs Fashion-MNIST from 70,000 fashion-product thumbnails and converts them into 28 × 28 grayscale images matching MNIST’s format.
Results
GradientBoostingClassifier achieved 0.880 test accuracy on Fashion-MNIST and 0.969 on MNIST in the reported benchmark.
Takeaways & Limitations
Fashion-MNIST provides an immediately compatible, more challenging alternative for benchmarking machine learning algorithms.
Abstract
from arXiv · showhide
We present Fashion-MNIST, a new dataset comprising of 28x28 grayscale images of 70,000 fashion products from 10 categories, with 7,000 images per category. The training set has 60,000 images and the test set has 10,000 images. Fashion-MNIST is intended to serve as a direct drop-in replacement for the original MNIST dataset for benchmarking machine learning algorithms, as it shares the same image size, data format and the structure of training and testing splits. The dataset is freely available at https://github.com/zalandoresearch/fashion-mnist
1 Introduction
Fashion-MNIST is introduced as an accessible benchmark dataset that preserves MNIST’s compact, convenient format while providing a more challenging classification task.
- Motivation: MNIST’s popularity stems from its small size and broad support across machine learning libraries and deep learning frameworks.These properties let researchers quickly check and prototype algorithms using built-in helpers and examples.
- Contribution: Fashion-MNIST retains MNIST’s 10 classes, 70,000 grayscale 28 × 28 images, straightforward encoding, and permissive license.The authors state that adopting it requires only changing the dataset URL.
- Contribution: Fashion-MNIST is designed as a drop-in benchmark replacement that preserves MNIST’s accessibility while posing a more challenging classification task.Its design aims to combine MNIST’s small size and usability with greater classification difficulty.
- Related work: Unlike EMNIST, Fashion-MNIST avoids requiring expanded framework helpers or neural-network changes for additional character classes.EMNIST introduces uppercase and lowercase characters, increasing the number of classes.
2 Fashion-MNIST Dataset
Fashion-MNIST is constructed from 70,000 Zalando product thumbnails across gender groups and converted into standardized 28×28 images. Products receive expert-reviewed silhouette-code labels, and the dataset uses a randomly selected 6,000 examples per class for training with MNIST-compatible file formats.
- Dataset construction: 70,000 unique Zalando products across men, women, kids, and neutral groups provide the front-look thumbnails used to build Fashion-MNIST.White products are excluded because they have low contrast with the background.
- Image preprocessing: The conversion pipeline trims background-like edges, resizes the longest edge to 28, sharpens pixels with a Gaussian operator, and extends the shortest edge to 28.Edge trimming uses a threshold within 5% of the maximum possible RGB intensity; sharpening uses radius and standard deviation 1.0.
- Class labeling: Class labels use each product’s single silhouette code, manually assigned by in-house fashion experts and reviewed by a separate Zalando team.Table 2 summarizes the class labels with example images.
- Dataset splits and storage: 6,000 examples from each class are randomly selected for training, and images and labels use the same file format as the MNIST dataset.Examples are sorted by label during storage, producing smaller compressed label files and facilitating example retrieval.
3 Experiments
The experiments establish a benchmark by averaging test-set accuracy over five shuffled-training repetitions and report side-by-side Fashion-MNIST and MNIST results. GradientBoostingClassifier and RandomForestClassifier achieve the strongest Fashion-MNIST accuracies among the reported classifiers.
- Benchmark protocol: All algorithms are repeated 5 times with shuffled training data, and average test-set accuracy is reported alongside MNIST results.The benchmark is presented in Table 3 for direct side-by-side comparison.
- Benchmark results: 0.880 Fashion-MNIST accuracy is achieved by GradientBoostingClassifier with n_estimators=100, loss=deviance, and max_depth=10.Its corresponding MNIST accuracy is 0.969.
- Benchmark results: 0.854 Fashion-MNIST accuracy is achieved by KNeighborsClassifier with distance weighting, n_neighbors=5, and p=1.Its corresponding MNIST accuracy is 0.959.
- Benchmark results: 0.842 Fashion-MNIST accuracy is achieved by LogisticRegression with C=1, multi_class=ovr, and penalty=l1.Its corresponding MNIST accuracy is 0.917.
- Benchmark results: 0.873 Fashion-MNIST accuracy is achieved by RandomForestClassifier with n_estimators=100, criterion=entropy, and max_depth=100.Its corresponding MNIST accuracy is 0.970.
4 Conclusions
Fashion-MNIST is introduced as a fashion-product image dataset designed to replace MNIST directly while providing a more challenging benchmark for machine-learning algorithms. Its MNIST-compatible format enables immediate use with packages supporting the original dataset.
- Conclusions: Fashion-MNIST is a fashion-product image dataset intended as a drop-in replacement for MNIST and a more challenging machine-learning benchmark.The dataset is designed for benchmarking machine-learning algorithms.
- Conclusions: The images are converted into a format matching MNIST, preserving compatibility with machine-learning packages that support the original dataset.This format makes Fashion-MNIST immediately usable with such packages.
- Conclusions: Fashion-MNIST’s compatibility is intended to allow immediate adoption without changing machine-learning packages designed for MNIST.The passage specifically states compatibility with any package capable of working with the original MNIST dataset.