Fruit freshness detection based on multi-task convolutional neural network

Yinsheng Zhang; Xudong Yang; Yongbo Cheng; Xiaojun Wu; Xiulan Sun; Ruiqi Hou; Haiyan Wang

doi:10.1016/j.crfs.2024.100733

Fruit freshness detection based on multi-task convolutional neural network

Curr Res Food Sci. 2024 Apr 8:8:100733. doi: 10.1016/j.crfs.2024.100733. eCollection 2024.

Authors

Yinsheng Zhang¹, Xudong Yang², Yongbo Cheng³, Xiaojun Wu⁴, Xiulan Sun⁵, Ruiqi Hou¹, Haiyan Wang¹

Affiliations

¹ Zhejiang Food and Drug Quality & Safety Engineering Research Institute, Zhejiang Gongshang University, Hangzhou, 310018, China.
² School of Management and E-Business, Zhejiang Gongshang University, Hangzhou, 310018, China.
³ School of Management Science and Engineering, Nanjing University of Finance and Economics, Nanjing, 210023, China.
⁴ Institute of Science and Technology, Jiangnan University, Wuxi, 214122, China.
⁵ School of Food Science and Technology, Jiangnan University, Wuxi, 214122, China.

Abstract

Background: Fruit freshness detection by computer vision is essential for many agricultural applications, e.g., automatic harvesting and supply chain monitoring. This paper proposes to use the multi-task learning (MTL) paradigm to build a deep convolutional neural work for fruit freshness detection.

Results: We design an MTL model that optimizes the freshness detection (T₁) and fruit type classification (T₂) tasks in parallel. The model uses a shared CNN (convolutional neural network) subnet and two FC (fully connected) task heads. The shared CNN acts as a feature extraction module and feeds the two task heads with common semantic features. Based on an open fruit image dataset, we conducted a comparative study of MTL and single-task learning (STL) paradigms. The STL models use the same CNN subnet with only one specific task head. In the MTL scenario, the T₁ and T₂ mean accuracies on the test set are 93.24% and 88.66%, respectively. Meanwhile, for STL, the two accuracies are 92.50% and 87.22%. Statistical tests report significant differences between MTL and STL on T1 and T2 test accuracies. We further investigated the extracted feature vectors (semantic embeddings) from the two STL models. The vectors have an averaged 0.7 cosine similarity on the entire dataset, with most values lying in the 0.6-0.8 range. This indicates a between-task correlation and justifies the effectiveness of the proposed MTL approach.

Conclusion: This study proves that MTL exploits the mutual correlation between two or more relevant tasks and can maximally share their underlying feature extraction process. we envision this approach to be extended to other domains that involve multiple interconnected tasks.

Keywords: Convolutional neural network; Depthwise separable convolution; Fruit freshness; Multi-task learning.