Dynamic detection of three-dimensional crop phenotypes based on a consumer-grade RGB-D camera

Peng Song; Zhengda Li; Meng Yang; Yang Shao; Zhen Pu; Wanneng Yang; Ruifang Zhai

doi:10.3389/fpls.2023.1097725

Dynamic detection of three-dimensional crop phenotypes based on a consumer-grade RGB-D camera

Front Plant Sci. 2023 Jan 27:14:1097725. doi: 10.3389/fpls.2023.1097725. eCollection 2023.

Authors

Peng Song¹, Zhengda Li¹, Meng Yang¹, Yang Shao¹, Zhen Pu¹, Wanneng Yang¹, Ruifang Zhai²

Affiliations

¹ National Key Laboratory of Crop Genetic Improvement, National Center of Plant Gene Research (Wuhan), Huazhong Agricultural University, Wuhan, China.
² College of Informatics, Huazhong Agricultural University, Wuhan, China.

Abstract

Introduction: Nondestructive detection of crop phenotypic traits in the field is very important for crop breeding. Ground-based mobile platforms equipped with sensors can efficiently and accurately obtain crop phenotypic traits. In this study, we propose a dynamic 3D data acquisition method in the field suitable for various crops by using a consumer-grade RGB-D camera installed on a ground-based movable platform, which can collect RGB images as well as depth images of crop canopy sequences dynamically.

Methods: A scale-invariant feature transform (SIFT) operator was used to detect adjacent date frames acquired by the RGB-D camera to calculate the point cloud alignment coarse matching matrix and the displacement distance of adjacent images. The data frames used for point cloud matching were selected according to the calculated displacement distance. Then, the colored ICP (iterative closest point) algorithm was used to determine the fine matching matrix and generate point clouds of the crop row. The clustering method was applied to segment the point cloud of each plant from the crop row point cloud, and 3D phenotypic traits, including plant height, leaf area and projected area of individual plants, were measured.

Results and discussion: We compared the effects of LIDAR and image-based 3D reconstruction methods, and experiments were carried out on corn, tobacco, cottons and Bletilla striata in the seedling stage. The results show that the measurements of the plant height (R²= 0.9~0.96, RSME = 0.015~0.023 m), leaf area (R²= 0.8~0.86, RSME = 0.0011~0.0041 m ² ) and projected area (R² = 0.96~0.99) have strong correlations with the manual measurement results. Additionally, 3D reconstruction results with different moving speeds and times throughout the day and in different scenes were also verified. The results show that the method can be applied to dynamic detection with a moving speed up to 0.6 m/s and can achieve acceptable detection results in the daytime, as well as at night. Thus, the proposed method can improve the efficiency of individual crop 3D point cloud data extraction with acceptable accuracy, which is a feasible solution for crop seedling 3D phenotyping outdoors.

Keywords: RGB-D camera; crop seedling detection; dynamic 3D reconstruction; field crop phenotype; point cloud segmentation.