Source-linked AI summary
DeepFood: Deep Learning-Based Food Image Recognition for Computer-Aided Dietary Assessment
Chang Liu, Yu Cao, Yan Luo, Guanling Chen, Vinod Vokkarane, Yunsheng Ma
TL;DR
Dietary assessment is hindered by reliance on memory and self-report, motivating automated analysis of food images captured by mobile devices. The paper proposes CNN-based food-image recognition and evaluates it on UEC-256 and Food-101, reporting performance exceeding existing approaches on both datasets.
Problem
Dietary assessment relies on memory and self-report, while deriving food information from images effectively and efficiently remains challenging.
Method
The paper proposes CNN-based food-image recognition algorithms, including an optimized model and convolution technique, and applies them to two public datasets.
Results
The proposed approach outperformed existing techniques on the UEC-100/UEC-256 and Food-101 datasets.
Takeaways & Limitations
The experiments support the proposed approach as a promising solution for food-image recognition and computer-aided dietary assessment.
Abstract
from arXiv · showhide
Worldwide, in 2014, more than 1.9 billion adults, 18 years and older, were overweight. Of these, over 600 million were obese. Accurately documenting dietary caloric intake is crucial to manage weight loss, but also presents challenges because most of the current methods for dietary assessment must rely on memory to recall foods eaten. The ultimate goal of our research is to develop computer-aided technical solutions to enhance and improve the accuracy of current measurements of dietary intake. Our proposed system in this paper aims to improve the accuracy of dietary assessment by analyzing the food images captured by mobile devices (e.g., smartphone). The key technique innovation in this paper is the deep learning-based food image recognition algorithms. Substantial research has demonstrated that digital imaging accurately estimates dietary intake in many environments and it has many advantages over other methods. However, how to derive the food information (e.g., food type and portion size) from food image effectively and efficiently remains a challenging and open research problem. We propose a new Convolutional Neural Network (CNN)-based food image recognition algorithm to address this problem. We applied our proposed approach to two real-world food image data sets (UEC-256 and Food-101) and achieved impressive results. To the best of our knowledge, these results outperformed all other reported work using these two data sets. Our experiments have demonstrated that the proposed approach is a promising solution for addressing the food image recognition problem. Our future work includes further improving the performance of the algorithms and integrating our system into a real-world mobile and cloud computing-based system to enhance the accuracy of current measurements of dietary intake.
1 Introduction
Dietary assessment relies on self-reported intake, which can be biased and burdensome. Mobile devices and cloud computing are proposed to capture and analyze dietary information more objectively.
- Accurate caloric-intake estimation is important for evaluating weight-loss interventions.
- Dietary assessment commonly relies on self-report and manually recorded instruments, including 24-hour recalls and food-frequency questionnaires.
- Self-estimation can bias dietary reporting through underreporting and underestimating food intake.
- Mobile cloud computing could use smartphones to capture dietary information in natural environments and analyze it automatically.
- Existing dietary apps can require users to enter food and other information manually, limiting automation.
2 Related Work
Related work combines mobile health technologies with visual dietary analysis, but many systems still depend on manual input or manual food assessment.
- Mobile health software and hardware create opportunities to support dietary assessment and health-related decision making.
- Many dietary apps track intake, exercise, and cloud data but remain dependent on manually entered user information.
- Eatly uses food images to assign health ratings, but its community performs the ratings manually rather than through automated algorithms.
- Visual dietary-analysis research has explored recognizing food through image features and spatial relationships among ingredients.
3 Proposed Approach
The paper proposes supervised CNN-based food-image recognition, combining an optimized model and convolution technique with Inception-style modules and transfer learning.
- 3 Proposed Approach: The proposed approach uses CNN-based algorithms with an optimized model and optimized convolution technique for food-image recognition.
- 3.2 Proposed CNN-based Approach for Visual-based Food Image Recognition: The supervised-learning approach is grounded in CNN architectures including LeNet-5, AlexNet, and GoogleNet.
- 3.2 Proposed CNN-based Approach for Visual-based Food Image Recognition: CNN layers learn image representations through convolution, subsampling or pooling, feature-map processing, and fully connected classification.
- 3.2 Proposed CNN-based Approach for Visual-based Food Image Recognition: The optimized convolution performs dimension reduction and increases depth within the Inception-based architecture.
- 3.2 Proposed CNN-based Approach for Visual-based Food Image Recognition: Inception modules are connected through max-pooling layers, with concatenated outputs passed into subsequent modules.
- 3.2 Proposed CNN-based Approach for Visual-based Food Image Recognition: The implementation fine-tunes a pre-trained ImageNet GoogLeNet model on UEC-256 and Food-101 datasets using Caffe.
4. Experimental Results
Experiments evaluated the proposed approach on UEC and Food-101 datasets, comparing it with existing methods and preprocessing choices. The reported results show improved performance from fine-tuning and bounding-box preprocessing.
- Datasets: The experiments used UEC-100/UEC-256 and Food-101, two publicly available food-image datasets.Food-101 contains 101 categories and 101,000 images, with approximately 75% used for training and 25% for testing.
- UEC results: The proposed method outperformed existing methods on UEC-100 using the same dataset and division.The comparison included the original DeepFoodCam results.
- Food-101 results: The proposed method was better than all existing work on Food-101 using the same dataset and division.This comparison was reported against state-of-the-art techniques evaluated on Food-101.
- Food-101 results: Pre-trained models with domain-specific fine-tuning significantly boosted classification accuracy compared with non-fine-tuned training.The paper denotes fine-tuned methods with “ft” and non-fine-tuned methods with “no ft.”
- Bounding-box preprocessing: Bounding-box preprocessing significantly improved classification accuracy on the evaluated UEC subsets.Cropping to the food region removed abundant background information and produced a clearer image candidate for training and testing; Food-101 lacked bounding boxes.
- Implementation: Training time depended on hardware, image-candidate size, iteration count, and learning rate; GoogLeNet required 2 to 3 seconds per image for a forward-backward pass on a K40.The authors therefore used a pre-trained model rather than training from scratch.
5. Conclusion
The paper concludes that CNN-based food-image recognition algorithms improved results on two challenging datasets. The authors plan to improve performance and integrate the system with mobile devices and cloud computing.
- Future work: Future work will improve algorithm performance and integrate the system into real-world mobile and cloud-computing systems.The stated goal is to enhance current measurements of dietary caloric intake.
- Acknowledgments: The project received partial support from the U.S. National Science Foundation, while the views expressed are those of the authors.The acknowledgment lists Awards 1547428, 1541434, 1440737, and 1229213.