Unstructured road extraction and roadside fruit recognition in grape orchards based on a synchronous detection algorithm

Xinzhao Zhou; Xiangjun Zou; Wei Tang; Zhiwei Yan; Hewei Meng; Xiwen Luo

doi:10.3389/fpls.2023.1103276

Unstructured road extraction and roadside fruit recognition in grape orchards based on a synchronous detection algorithm

Front Plant Sci. 2023 Jun 2:14:1103276. doi: 10.3389/fpls.2023.1103276. eCollection 2023.

Authors

Xinzhao Zhou^{1

2}, Xiangjun Zou^{2

3}, Wei Tang², Zhiwei Yan², Hewei Meng¹, Xiwen Luo^{1

4

5}

Affiliations

¹ College of Mechanical and Electrical Engineering, Shihezi University, Shihezi, China.
² Foshan-Zhongke Innovation Research Institute of Intelligent Agriculture, Foshan, China.
³ Foshan Sino-tech Industrial Technology Research Institute, Foshan, China.
⁴ College of Engineering, South China Agricultural University, Guangzhou, China.
⁵ Guangdong Provincial Key Laboratory of Agricultural Artificial Intelligence (GDKL-AAI), Guangzhou, China.

Abstract

Accurate road extraction and recognition of roadside fruit in complex orchard environments are essential prerequisites for robotic fruit picking and walking behavioral decisions. In this study, a novel algorithm was proposed for unstructured road extraction and roadside fruit synchronous recognition, with wine grapes and nonstructural orchards as research objects. Initially, a preprocessing method tailored to field orchards was proposed to reduce the interference of adverse factors in the operating environment. The preprocessing method contained 4 parts: interception of regions of interest, bilateral filter, logarithmic space transformation and image enhancement based on the MSRCR algorithm. Subsequently, the analysis of the enhanced image enabled the optimization of the gray factor, and a road region extraction method based on dual-space fusion was proposed by color channel enhancement and gray factor optimization. Furthermore, the YOLO model suitable for grape cluster recognition in the wild environment was selected, and its parameters were optimized to enhance the recognition performance of the model for randomly distributed grapes. Finally, a fusion recognition framework was innovatively established, wherein the road extraction result was taken as input, and the optimized parameter YOLO model was utilized to identify roadside fruits, thus realizing synchronous road extraction and roadside fruit detection. Experimental results demonstrated that the proposed method based on the pretreatment could reduce the impact of interfering factors in complex orchard environments and enhance the quality of road extraction. Using the optimized YOLOv7 model, the precision, recall, mAP, and F1-score for roadside fruit cluster detection were 88.9%, 89.7%, 93.4%, and 89.3%, respectively, all of which were higher than those of the YOLOv5 model and were more suitable for roadside grape recognition. Compared to the identification results obtained by the grape detection algorithm alone, the proposed synchronous algorithm increased the number of fruit identifications by 23.84% and the detection speed by 14.33%. This research enhanced the perception ability of robots and provided a solid support for behavioral decision systems.

Keywords: deep learning; fruit harvesting robot; machine vision; non-structural environment; roadside fruits detection.

Grants and funding

This study was supported by the scientific and technological breakthrough plan in key fields of Xinjiang Production and Construction Corps, China (Approval No.2022DB004), Foshan Science and Technology Innovation Project, China (Approval No.2120001008424) and Guangdong Basic and Applied Basic Research Foundation (Approval No.2022A1515140162 and No.2022A1515140013).