Detection and Reconstruction of Passion Fruit Branches via CNN and Bidirectional Sector Search

Jiangchuan Bao; Guo Li; Haolan Mo; Tingting Qian; Ming Chen; Shenglian Lu

doi:10.34133/plantphenomics.0088

Detection and Reconstruction of Passion Fruit Branches via CNN and Bidirectional Sector Search

Plant Phenomics. 2023 Sep 8:5:0088. doi: 10.34133/plantphenomics.0088. eCollection 2023.

Authors

Jiangchuan Bao¹, Guo Li¹, Haolan Mo², Tingting Qian³, Ming Chen^{1

4}, Shenglian Lu^{1

4}

Affiliations

¹ Key Lab of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University, Guilin 541004, China.
² Guilin Center for Agricultural Science & Technology Research, Guilin 541006, China.
³ Agricultural Information Institutes of Science and Technology, Shanghai Academy of Agriculture Sciences, Shanghai 201403, China.
⁴ Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University, Guilin, 541004, China.

Abstract

Accurate detection and reconstruction of branches aid the accuracy of harvesting robots and extraction of plant phenotypic information. However, the complex orchard background and twisting growing branches of vine fruit trees make this challenging. To solve these problems, this study adopted a Mask Region-based convolutional neural network (Mask R-CNN) architecture incorporating deformable convolution to segment branches in complex backgrounds. Based on the growth posture, a branch reconstruction algorithm with bidirectional sector search was proposed to adaptively reconstruct the segmented branches obtained by an improved model. The average precision, average recall, and F1 scores of the improved Mask R-CNN model for passion fruit branch detection were found to be 64.30%, 76.51%, and 69.88%, respectively, and the average running time on the test dataset was 0.75 s per image, which is better than the compared model. We randomly selected 40 images from the test dataset to evaluate the branch reconstruction. The branch reconstruction accuracy, average error, average relative error of reconstructed diameter, and mean intersection-over-union (mIoU) were 88.83%, 1.98 px, 7.98, and 83.44%, respectively. The average reconstruction time for a single image was 0.38 s. This would promise the proposed method to detect and reconstruct plant branches under complex orchard backgrounds.