An improved YOLOv5s model for assessing apple graspability in automated harvesting scene

Huibin Li; Peng Yang; Huaiyang Liu; Xiang Liu; Jianping Qian; Qiangyi Yu; Changxing Geng; Yun Shi

doi:10.3389/fpls.2023.1323453

An improved YOLOv5s model for assessing apple graspability in automated harvesting scene

Front Plant Sci. 2023 Dec 11:14:1323453. doi: 10.3389/fpls.2023.1323453. eCollection 2023.

Authors

Huibin Li¹, Peng Yang², Huaiyang Liu³, Xiang Liu³, Jianping Qian¹, Qiangyi Yu¹, Changxing Geng³, Yun Shi¹

Affiliations

¹ State Key Laboratory of Efficient Utilization of Arid and Semi-arid Arable Land in Northern China, Institute of Agricultural Resources and Regional Planning, Chinese Academy of Agricultural Sciences, Beijing, China.
² Agricultural Algorithm Research Department, Suzhou Zhongnong Digital Intelligence Technology Co., Ltd, Suzhou, China.
³ School of Mechanical and Electrical Engineering, Soochow University, Suzhou, China.

Abstract

Introduction: With continuously increasing labor costs, an urgent need for automated apple- Qpicking equipment has emerged in the agricultural sector. Prior to apple harvesting, it is imperative that the equipment not only accurately locates the apples, but also discerns the graspability of the fruit. While numerous studies on apple detection have been conducted, the challenges related to determining apple graspability remain unresolved.

Methods: This study introduces a method for detecting multi-occluded apples based on an enhanced YOLOv5s model, with the aim of identifying the type of apple occlusion in complex orchard environments and determining apple graspability. Using bootstrap your own atent(BYOL) and knowledge transfer(KT) strategies, we effectively enhance the classification accuracy for multi-occluded apples while reducing data production costs. A selective kernel (SK) module is also incorporated, enabling the network model to more precisely identify various apple occlusion types. To evaluate the performance of our network model, we define three key metrics: AP_GA, AP_TUGA, and AP_UGA, representing the average detection accuracy for graspable, temporarily ungraspable, and ungraspable apples, respectively.

Results: Experimental results indicate that the improved YOLOv5s model performs exceptionally well, achieving detection accuracies of 94.78%, 93.86%, and 94.98% for AP_GA, AP_TUGA, and AP_UGA, respectively.

Discussion: Compared to current lightweight network models such as YOLOX-s and YOLOv7s, our proposed method demonstrates significant advantages across multiple evaluation metrics. In future research, we intend to integrate fruit posture and occlusion detection to f]urther enhance the visual perception capabilities of apple-picking equipment.

Keywords: BYOL; YOLOv5S; apple harvesting; attention mechanism; occlusion detection.

Grants and funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported by the following projects: research and development of key technologies for agricultural situation parameter acquisition and integrated application of sensing equipment-2022LQ02004 and National Natural Science Foundation of China-Regional Innovation and Development Joint Fund (Jilin) U19A2061.