DeepTrayMeal: Automatic dietary assessment for Chinese tray meals based on deep learning

Jialin Shi; Qi Han; Zhongxiang Cao; Zongjie Wang

doi:10.1016/j.foodchem.2023.137525

DeepTrayMeal: Automatic dietary assessment for Chinese tray meals based on deep learning

Food Chem. 2024 Feb 15:434:137525. doi: 10.1016/j.foodchem.2023.137525. Epub 2023 Sep 18.

Authors

Jialin Shi¹, Qi Han¹, Zhongxiang Cao¹, Zongjie Wang²

Affiliations

¹ School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, China.
² School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, China. Electronic address: Wangzj@ustb.edu.cn.

PMID: 37742550
DOI: 10.1016/j.foodchem.2023.137525

Abstract

Tray meal is a popular way of eating in China, and tray-based automatic dietary assessment is important for public health. Relevant research is lacking because public tray meal datasets and suitable methods are unavailable. In this study, we established and published the first Chinese tray meal dataset, the ChinaLunchTray-99. We collected real-world 1185 tray meal images, covering 99 dish categories with corresponding manually annotated bounding box and category-level labels. We developed a new framework for automatic dietary assessment, which consists of dish image recognition, volume estimation and nutrition mapping. First, we demonstrated a tray meal detection model considering feature extraction, anchor scales, and loss function, resulting in a high mean Average Precision of 92.13%. Second, we proposed an automatic method to estimate volume via detection results and tray's information. Finally, nutrients were mapped from the estimated volume. Our research can promote applications of automatic dietary assessment for Chinese tray meals.

Keywords: Chinese tray meals; Deep learning; Nutrition estimation; Tray meal detection.