Cotton plant part 3D segmentation and architectural trait extraction using point voxel convolutional neural networks

Farah Saeed; Shangpeng Sun; Javier Rodriguez-Sanchez; John Snider; Tianming Liu; Changying Li

doi:10.1186/s13007-023-00996-1

Cotton plant part 3D segmentation and architectural trait extraction using point voxel convolutional neural networks

Plant Methods. 2023 Mar 30;19(1):33. doi: 10.1186/s13007-023-00996-1.

Authors

Farah Saeed¹, Shangpeng Sun², Javier Rodriguez-Sanchez³, John Snider⁴, Tianming Liu¹, Changying Li^{5

6

7}

Affiliations

¹ School of Computing, University of Georgia, Athens, Georgia, USA.
² Department of Bioresource Engineering, McGill University, Montreal, Canada.
³ College of Engineering, University of Georgia, Athens, Georgia, USA.
⁴ Department of Crop and Soil Science, University of Georgia, Tifton, Georgia, USA.
⁵ School of Computing, University of Georgia, Athens, Georgia, USA. cli2@ufl.edu.
⁶ College of Engineering, University of Georgia, Athens, Georgia, USA. cli2@ufl.edu.
⁷ Bio-Sensing, Automation & Intelligence Laboratory, Agricultural and Biological Engineering Department, University of Florida, 223 Frazier Rogers Hall, 1741 Museum Road, PO Box 110570, Gainesville, Florida, 32611-0570, USA. cli2@ufl.edu.

Abstract

Background: Plant architecture can influence crop yield and quality. Manual extraction of architectural traits is, however, time-consuming, tedious, and error prone. The trait estimation from 3D data addresses occlusion issues with the availability of depth information while deep learning approaches enable learning features without manual design. The goal of this study was to develop a data processing workflow by leveraging 3D deep learning models and a novel 3D data annotation tool to segment cotton plant parts and derive important architectural traits.

Results: The Point Voxel Convolutional Neural Network (PVCNN) combining both point- and voxel-based representations of 3D data shows less time consumption and better segmentation performance than point-based networks. Results indicate that the best mIoU (89.12%) and accuracy (96.19%) with average inference time of 0.88 s were achieved through PVCNN, compared to Pointnet and Pointnet++. On the seven derived architectural traits from segmented parts, an R² value of more than 0.8 and mean absolute percentage error of less than 10% were attained.

Conclusion: This plant part segmentation method based on 3D deep learning enables effective and efficient architectural trait measurement from point clouds, which could be useful to advance plant breeding programs and characterization of in-season developmental traits. The plant part segmentation code is available at https://github.com/UGA-BSAIL/plant_3d_deep_learning .

Keywords: 3D Deep learning; LiDAR; Plant architecture; Plant part segmentation; Point cloud.

Grants and funding

1934481/National Science Foundation