Source-linked AI summary
ChineseFoodNet: A large-scale Image Dataset for Chinese Food Recognition
Xin Chen, Yu Zhu, Hua Zhou, Liang Diao, Dongyan Wang
TL;DR
Chinese dish recognition is challenging because dishes vary within categories and occur in complex backgrounds. The paper builds ChineseFoodNet and proposes TastyNet, which fuses CNN predictions by voting. TastyNet reports 81.43% top-1 accuracy on validation and 81.55% on test, improving over the best single-network approaches.
Problem
Chinese dish recognition is challenging because same-category dishes vary in ingredients and cooking methods and appear in complex backgrounds.
Source: §I · p. 1Method
The paper builds ChineseFoodNet from web and unconstrained real-world images and proposes TastyNet, which fuses selected CNN predictions by voting.
Source: §I · p. 2Results
81.43% top-1 accuracy on the validation set and 81.55% on the test set; improvements over the best single-network approaches were 2.38% and 2.33%, respectively.
Source: §I · p. 2Takeaways & Limitations
In this evaluation, combinations of different CNN architectures improved performance more than same-architecture combinations, and deeper, wider combinations obtained the best results.
Source: B. Results and Analysis · p. 6
Abstract
from arXiv · showhide
In this paper, we introduce a new and challenging large-scale food image dataset called "ChineseFoodNet", which aims to automatically recognizing pictured Chinese dishes. Most of the existing food image datasets collected food images either from recipe pictures or selfie. In our dataset, images of each food category of our dataset consists of not only web recipe and menu pictures but photos taken from real dishes, recipe and menu as well. ChineseFoodNet contains over 180,000 food photos of 208 categories, with each category covering a large variations in presentations of same Chinese food. We present our efforts to build this large-scale image dataset, including food category selection, data collection, and data clean and label, in particular how to use machine learning methods to reduce manual labeling work that is an expensive process. We share a detailed benchmark of several state-of-the-art deep convolutional neural networks (CNNs) on ChineseFoodNet. We further propose a novel two-step data fusion approach referred as "TastyNet", which combines prediction results from different CNNs with voting method. Our proposed approach achieves top-1 accuracies of 81.43% on the validation set and 81.55% on the test set, respectively. The latest dataset is public available for research and can be achieved at https://sites.google.com/view/chinesefoodnet.
I. INTRODUCTION
Chinese dish recognition is difficult because dishes vary within categories and appear in complex, unconstrained settings. The paper introduces ChineseFoodNet, benchmarks CNNs on it, and proposes TastyNet, a voting-based fusion method.
I. INTRODUCTION
- Chinese dish recognition is challenging because same-category dishes vary in ingredients and cooking methods and appear against complex backgrounds.The paper notes that these differences can make images visually distinct even to human observers.Source: §I · p. 1
- ChineseFoodNet contains 185,628 images across 208 popular Chinese food categories, including web images and unconstrained real-world photos.The dataset was built to address the difficulty of recognizing visually varied dishes in complex backgrounds.Source: §I · p. 2
- The paper benchmarks nine CNN models across four architectures and reports that ChineseFoodNet can support learning complex models.The evaluated architectures are SqueezeNet, VGG, ResNet, and DenseNet.Source: §I · p. 2
- 81.43% top-1 accuracy on the validation set and 81.55% on the test set; TastyNet improved 2.38% and 2.33%, respectively, over the best single-network approaches.TastyNet fuses predictions from ResNet152, DenseNet121, DenseNet169, DenseNet201, and VGG19-BN by voting.Source: §I · p. 2
A. Food Dataset
Existing food datasets span varied cuisines and tasks, while prior visual-recognition methods range from handcrafted features to CNNs. The paper identifies a gap: these approaches had not been tested on a large-scale Chinese food image dataset.
A. Food Dataset
- Food datasets include recipe images, selfies, Japanese food collections, and Chinese datasets designed for tasks such as ingredient recognition or recipe retrieval.The cited examples include PFID, UNICT-FD889, UEC-Food100/256, UPMC-FOOD-101, ETHZ-FOOD-101, and VIREO-172.Source: A. Food Dataset · p. 2
B. Visual Dish Recognition
- Visual food recognition has used handcrafted features, but CNN models have outperformed traditional methods and improved with increasing depth.Examples of traditional methods include Gabor filters, bag-of-words, random forests, and Fisher vectors.Source: B. Visual Dish Recognition · p. 2
- Neither traditional nor deep-learning approaches had been tested on a large-scale Chinese food image dataset.Source: B. Visual Dish Recognition · p. 2
III. CHINESEFOODNET: A LARGE-SCALE CHINESE FOOD IMAGE DATASET
The authors present ChineseFoodNet as a large-scale resource for training data-driven Chinese dish recognition models, using a semi-supervised method to accelerate labeling. Their category selection aims to cover popular Chinese cuisines and consolidate visually similar dishes.
III. CHINESEFOODNET: A LARGE-SCALE CHINESE FOOD IMAGE DATASET
- The paper introduces a semi-supervised method to accelerate dataset labeling, which is costly when validating large numbers of images.Source: §III · p. 2
- ChineseFoodNet’s 185,628 color images are organized into 208 categories.Figure 2 presents fifty sample images; they were resized for presentation.Source: Figure 2 · p. 3
A. Category Selection
- The authors selected categories to cover popular Chinese cuisines, combining internet searches with a group survey and merging visually similar dishes into 208 categories.They initially gathered 250 categories from the internet and selected about 300 after the survey before merging related categories.Source: A. Category Selection · p. 3
B. Data Collection
ChineseFoodNet combines web images and photos taken in daily life, then uses automated filtering and label-assistance steps to clean the collected images. The process addresses both varied image quality and the expense of manual label validation.
B. Data Collection
- The image sources are Chinese food social-network uploads and photos collected by the authors in daily life; the combined collection exceeds 500,000 images.The collected images may have missing, incorrect, or unclear labels.Source: B. Data Collection · p. 3
C. Data Clean and Label
- Cleaning removes irregular-sized and low-entropy images, then removes near-duplicates using AlexNet features and Euclidean distance.The process computes 1,024 features from AlexNet’s last fully connected layer and removes one image when the distance falls below a threshold.Source: C. Data Clean and Label · p. 3
- Recipe and menu images often have directly extractable labels, but are limited in number and differ in distribution from daily-life images.The paper notes that their typically high image quality could affect recognition tasks in real-life settings.Source: C. Data Clean and Label · p. 3
- For other images, manually generated class keywords match metadata to select and assign labels.The metadata is associated with user-uploaded food photos and can describe the food and its recipe.Source: C. Data Clean and Label · pp. 3–4
- A shallow CNN proposes top-five candidate labels for collected images, which people then validate to remove incorrect labels.The CNN is trained on a small, separately crowdsourced food-image database with the same class labels.Source: C. Data Clean and Label · p. 4
D. Dataset Description
The completed ChineseFoodNet dataset contains 185,628 images across 208 food categories, split into training, validation, and test sets.
D. Dataset Description
- The dataset description follows the authors’ category selection, collection, and cleaning steps.Source: D. Dataset Description · p. 4
- ChineseFoodNet contains 185,628 images across 208 food categories and occupies 19.4 GB.Each image has one label, numbered 0 to 207.Source: D. Dataset Description · p. 4
- The dataset is divided into 145,066 training, 20,254 validation, and 20,310 test images, approximately an 80/10/10 split.The images retain their original size and color without processing.Source: D. Dataset Description · p. 4
IV. BENCHMARK ON CHINESEFOODNET DATASET
The benchmark evaluates CNNs on ChineseFoodNet using a shared training and testing procedure. It reports recognition rates using top-1 and top-5 accuracy on validation and test sets.
A. Experimental settings
- Training uses stochastic gradient descent with momentum for 90 epochs, with the learning rate decayed by a factor of 10 every 30 epochs.The initial learning rate is 0.01, momentum is 0.9, and weight decay is 1e-4.Source: A. Experimental settings · p. 4
IV. BENCHMARK ON CHINESEFOODNET DATASET
- The benchmark presents top-1 and top-5 accuracy on both validation and test sets.Table I is identified as the recognition-rate table for the food dataset.Source: Table I · p. 5
A. Experimental settings
- Images are resized to 256x256 and cropped to 224x224; training uses random crops and horizontal flips, while testing uses center crops.The networks are fine-tuned from ImageNet-pretrained models.Source: A. Experimental settings · p. 5
B. Experimental Protocol
The benchmark compares several CNN architectures under shared image-input and preprocessing conditions. DenseNet201 has the best reported validation top-1 accuracy, while VGG19 leads on test top-1 accuracy.
B. Experimental Protocol
- The protocol benchmarks SqueezeNet v1.1, VGG19-BN, ResNet variants, and DenseNet variants using the same input size and preprocessing and postprocessing procedures.The dataset is split by random selection into training, validation, and test sets.Source: B. Experimental Protocol · p. 5
- Figure 4 illustrates the residual block in ResNet and the fire block in SqueezeNet.Source: Figure 4 · p. 5
C. Experimental Results
- 79.05% validation top-1 accuracy is achieved by DenseNet201, while 79.22% test recognition rate is achieved by VGG19.Resnet152 is second on the test set, 0.22% below VGG19; the results are presented alongside top-1 and top-5 accuracy.Source: C. Experimental Results · p. 5
- Deeper models generally perform better within the same architecture; ResNet-152 improves by about 5% over ResNet-18 on both validation and test sets.DenseNet shows a similar pattern, while SqueezeNet v1.1 has the lowest reported results: 58.42% validation and 58.24% test.Source: C. Experimental Results · p. 5
V. TASTYNET: A TWO-STEP DATA FUSION APPROACH
TastyNet fuses predictions from multiple CNNs because deeper models bring accuracy gains but also greater computation, memory use, and overfitting risk. The selected ensemble averages the outputs of five models.
V. TASTYNET: A TWO-STEP DATA FUSION APPROACH
- Deeper CNNs improve accuracy, but require more computation and memory and can more easily overfit, motivating fusion as an alternative.The authors frame fusion as combining inference results from different models.Source: A. Methodology · p. 5
- TastyNet gathers predictions from different networks and uses voting to produce one final prediction.The paper describes its voting policy as using weighted coefficients of the first-step results.Source: A. Methodology · p. 5
- The selected combination uses ResNet152, DenseNet121, DenseNet169, DenseNet201, and VGG19-BN, chosen for the best top-1 result among tested combinations.The method averages the results of all selected models.Source: A. Methodology · p. 5
B. Results and Analysis
Across tested fusion schemes, combining different CNN architectures performs better than same-architecture combinations. The best reported ensemble achieves 81.43% validation and 81.55% test accuracy.
B. Results and Analysis
- The fusion scheme gathers predictions from ResNet152, DenseNet121, DenseNet169, DenseNet201, and VGG19-BN, then combines them with voting.The figure caption specifies a weighted coefficient for combining first-step results.Source: Figure 5 · p. 6
- The algorithm averages the models’ class prediction results and outputs the class index with the maximum average.The listed class indices range from 0 to 207.Source: B. Results and Analysis · p. 6
- 81.43% validation accuracy and 81.55% test accuracy are reported for the best fusion, 2.38% and 2.33% above the single network, respectively.The ensemble combines ResNet152, DenseNet121, DenseNet169, DenseNet201, and VGG19-BN.Source: B. Results and Analysis · p. 6, Table II · p. 6
- Different-architecture combinations improve recognition more than same-architecture combinations, with deeper and wider networks giving the best evaluated results.The paper reports that fusion improves overall performance over a single deep network.Source: B. Results and Analysis · p. 6
VI. CONCLUSION AND FUTURE WORK
ChineseFoodNet provides a large-scale dataset for Chinese dish recognition, and TastyNet’s fusion approach achieves the paper’s reported state-of-the-art results on it. Future work targets more categories and new fusion methods.
VI. CONCLUSION AND FUTURE WORK
- ChineseFoodNet contains 185,628 images across 208 food categories, sourced from both web images and the real world.The authors also report benchmarks of nine CNN models spanning four architectures.Source: §VI · p. 6
- TastyNet fuses predictions from ResNet152, DenseNet121, DenseNet169, DenseNet201, and VGG19+BN, and the authors report state-of-the-art results on ChineseFoodNet.The paper concludes that fusion can outperform relying on one CNN model.Source: §VI · pp. 6–7
- Future work will extend the dataset to over 500 food categories and investigate new methods for fusing model results.Source: §VI · p. 7