Source-linked AI summary
ISIA Food-500: A Dataset for Large-Scale Food Recognition via Stacked Global-Local Attention Network
Weiqing Min, Linhu Liu, Zhiling Wang, Zhengdong Luo, Xiaoming Wei, Xiaolin Wei, Shuqiang Jiang
TL;DR
Food recognition needs larger, more diverse benchmarks and representations that capture both broad appearance and subtle local distinctions. The paper introduces ISIA Food-500 and SGLANet, whose joint global-local design is evaluated as an effective strong baseline across three datasets.
Problem
Food recognition lacks large-scale diverse datasets, and existing features struggle with substantial appearance variation and subtle fine-grained distinctions.
Method
The paper introduces ISIA Food-500 and SGLANet, which combines attention-enhanced global features with spatially transformed local regional features.
Results
About 0.9 percent higher Top-1 accuracy than SENet-154 was achieved on the ISIA Food-500 test set.
Takeaways & Limitations
Extensive evaluation on ISIA Food-500 and two other benchmark datasets supports SGLANet as one strong baseline for food recognition.
Takeaways & Limitations
The dataset is still being expanded toward about 1.5 million images across about 2,000 categories, with richer attribute information planned for future multimodal recognition.
Abstract
from arXiv · showhide
Food recognition has received more and more attention in the multimedia community for its various real-world applications, such as diet management and self-service restaurants. A large-scale ontology of food images is urgently needed for developing advanced large-scale food recognition algorithms, as well as for providing the benchmark dataset for such algorithms. To encourage further progress in food recognition, we introduce the dataset ISIA Food- 500 with 500 categories from the list in the Wikipedia and 399,726 images, a more comprehensive food dataset that surpasses existing popular benchmark datasets by category coverage and data volume. Furthermore, we propose a stacked global-local attention network, which consists of two sub-networks for food recognition. One subnetwork first utilizes hybrid spatial-channel attention to extract more discriminative features, and then aggregates these multi-scale discriminative features from multiple layers into global-level representation (e.g., texture and shape information about food). The other one generates attentional regions (e.g., ingredient relevant regions) from different regions via cascaded spatial transformers, and further aggregates these multi-scale regional features from different layers into local-level representation. These two types of features are finally fused as comprehensive representation for food recognition. Extensive experiments on ISIA Food-500 and other two popular benchmark datasets demonstrate the effectiveness of our proposed method, and thus can be considered as one strong baseline. The dataset, code and models can be found at http://123.57.42.89/FoodComputing-Dataset/ISIA-Food500.html.
1 INTRODUCTION
Food recognition remains difficult because existing datasets lack scale and diversity, while food images vary substantially in global appearance and contain subtle fine-grained differences. The paper addresses these challenges with the ISIA Food-500 dataset and a stacked global-local attention network.
- Food recognition supports applications including visual food diaries, health-aware recommendation, and self-service restaurants.
- Existing food datasets lack broad category coverage and diversity, limiting their suitability for more complicated recognition models.ETH Food-101 has 101 categories and 101,000 images, while Vireo Food-172 covers 172 Chinese food categories.
- Food images exhibit large intra-class variation in appearance, shape, texture, and other configurations.
- Fine-grained food distinctions are often too subtle for global features or conventional CNNs, especially when contextual information is unavailable.
- ISIA Food-500 provides 399,726 images across 500 categories, while SGLANet jointly learns complementary global and local visual features.Its global subnetwork aggregates attention-enhanced multi-layer features, whereas its local subnetwork uses cascaded spatial transformers to aggregate regional features.
- The paper introduces a stacked global-local attention architecture and evaluates it on the proposed dataset and two other food benchmarks.The authors describe the approach as a strong baseline and plan to release code and models.
2 RELATED WORK
Prior food-recognition datasets and methods leave gaps in category coverage, multimodal scope, and food-specific feature representation. This work targets those gaps with a recognition dataset and a two-branch network that combines global and local food-oriented features.
- Existing food datasets include PFID, ETHZ Food-101, and VIREO Food-172, but earlier benchmarks provide less comprehensive food-category coverage.
- Recipe1M and related datasets pair recipes with images for cross-modal retrieval, whereas ISIA Food-500 targets multimedia food recognition.
- Food-recognition methods have progressed from hand-crafted features to deep learning and sometimes incorporate ingredients or restaurant information.
- The proposed work avoids additional context information and jointly learns global texture and shape features with local ingredient-relevant regional features.
- Because food recognition is fine-grained, the network is designed around characteristics specific to food images.
3 ISIA FOOD-500
ISIA Food-500 was constructed through category-list design, diverse image collection, automated and manual cleaning, and dataset scaling. The resulting dataset contains 399,726 images in 500 categories with broad culinary coverage and higher diversity than earlier benchmarks.
- The dataset construction process comprises category-list construction, image collection, image cleaning and preprocessing, and scaling up the dataset.
- Wikipedia food lists and depth-first search were used to build a broad food concept system before redundant categories were removed.
- Images were crawled from Google, Bing, and Baidu using expanded food-related queries to increase coverage and diversity.
- Automatic cleaning removed invalid or small images and non-food images, followed by manual verification by 20 laboratory members.A VGG-16 food/non-food classifier achieved 99.48% accuracy on its test set.
- Additional images were gathered through multilingual category queries and recipe or food-sharing websites to expand underrepresented categories.
- 399,726 images across 500 categories yield about 800 images per category and broader coverage of eastern and western cuisines.The dataset is described as exceeding ETH Food-101 and Vireo Food-172 in data volume, category coverage, and diversity.
4 FRAMEWORK
SGLANet jointly learns complementary global and local food features through attention, multi-layer fusion, spatial transformers, and multiple losses. Its global branch captures shape, texture, and edge cues, while its local branch aggregates regional features from different positions and scales.
- Framework overview: SGLANet jointly learns complementary global and local features through the GloFLS and LocFLS subnetworks.The framework fuses both feature types for food recognition and trains them end-to-end with global, local, and joint losses.
- Global Feature Learning Sub-network: GloFLS applies hybrid spatial-channel attention at multiple layers and fuses the resulting features into a global representation.The multi-layer representation captures low-, mid-, and high-level cues, including food shape, texture, and edges.
- Global Feature Learning Sub-network: The SCA module produces spatial and channel attention maps for each layer, yielding saliency weights over feature locations and channels.Its input is a 3-D feature tensor, and spatial and channel attention are computed using global average pooling and a squeeze-and-excitation block.
- Local Feature Learning Sub-network: LocFLS uses stacked spatial transformers to localize discriminative regions at different positions and scales, then fuses regional features across layers.Inception blocks, global average pooling, max pooling, concatenation, and a fully connected layer produce the final local features.
- Learning with Multiple Losses: SGLANet optimizes joint, global, and local cross-entropy losses with balance parameters to maximize complementary discriminative information.The final objective combines the three losses as L = L_Joi + γ1L_Glo + γ2L_Loc.
5 EXPERIMENT
Experiments evaluate SGLANet through ablations, comparisons, visualizations, and tests on additional food benchmarks. Results support complementary global-local learning, while performance remains uneven across categories.
- Experimental Setup: A 60%/10%/30% train-validation-test split and single centered crop define the default ISIA Food-500 evaluation setting.
- Ablation Study: Combining SCA with multi-scale features improves recognition over either component alone, indicating complementary effects within GloFLS.
- Ablation Study: Joining global and local representations increases Top-1 accuracy, supporting complementary feature learning across GloFLS and LocFLS.
- Ablation Study: Different losses together produce the best recognition performance, whereas adding only one loss does not improve the baseline without both global and local losses.
- Comparisons with State-of-the-Art: SGLANet exceeds all evaluated state-of-the-art baselines in Top-1 and Top-5 accuracy, improving Top-1 accuracy by about 0.9 percent over SENet-154.
- Visualization of GloFLS and LocFLS: Visualizations show deeper SCA layers becoming more focused and discriminative while spatial transformers select local regions with less background.
- Qualitative Analysis: Top-1 accuracy exceeds 97% for Chakli and Edamame but remains below 10% for Curry_rice and kebab, with small inter-class variations identified as the main difficulty.
6 CONCLUSIONS
The paper introduces ISIA Food-500 and a stacked global-local attention network for food recognition, with evaluation across three datasets supporting its effectiveness as a strong baseline. Future work expands the dataset substantially and adds richer food attributes.
- ISIA Food-500 provides larger data volume, broader category coverage, and higher diversity than existing typical food datasets.
- The stacked global-local attention network jointly exploits complementary global and local features through two designed subnetworks.
- Extensive evaluation on ISIA Food-500 and two additional benchmark datasets verifies the network’s effectiveness as a strong baseline.
- Future work targets about 1.5 million food images across about 2,000 categories and richer attribute information for multimodal food recognition.