Drought stress prediction and propagation using time series modeling on multimodal plant image sequences

Sruti Das Choudhury; Sinjoy Saha; Ashok Samal; Anastasios Mazis; Tala Awada

doi:10.3389/fpls.2023.1003150

Drought stress prediction and propagation using time series modeling on multimodal plant image sequences

Front Plant Sci. 2023 Feb 9:14:1003150. doi: 10.3389/fpls.2023.1003150. eCollection 2023.

Authors

Sruti Das Choudhury^{1

2}, Sinjoy Saha³, Ashok Samal³, Anastasios Mazis^{1

4}, Tala Awada^{1

5}

Affiliations

¹ School of Natural Resources, University of Nebraska-Lincoln, Lincoln, NE, United States.
² School of Computing, University of Nebraska-Lincoln, Lincoln, NE, United States.
³ Institute of Radio Physics and Electronics, University of Calcutta, Kolkata, West Bengal, India.
⁴ Department of Civil and Environmental Engineering, University of California, Merced, Merced, CA, United States.
⁵ Agricultural Research Division, University of Nebraska-Lincoln, Lincoln, NE, United States.

Abstract

The paper introduces two novel algorithms for predicting and propagating drought stress in plants using image sequences captured by cameras in two modalities, i.e., visible light and hyperspectral. The first algorithm, VisStressPredict, computes a time series of holistic phenotypes, e.g., height, biomass, and size, by analyzing image sequences captured by a visible light camera at discrete time intervals and then adapts dynamic time warping (DTW), a technique for measuring similarity between temporal sequences for dynamic phenotypic analysis, to predict the onset of drought stress. The second algorithm, HyperStressPropagateNet, leverages a deep neural network for temporal stress propagation using hyperspectral imagery. It uses a convolutional neural network to classify the reflectance spectra at individual pixels as either stressed or unstressed to determine the temporal propagation of stress in the plant. A very high correlation between the soil water content, and the percentage of the plant under stress as computed by HyperStressPropagateNet on a given day demonstrates its efficacy. Although VisStressPredict and HyperStressPropagateNet fundamentally differ in their goals and hence in the input image sequences and underlying approaches, the onset of stress as predicted by stress factor curves computed by VisStressPredict correlates extremely well with the day of appearance of stress pixels in the plants as computed by HyperStressPropagateNet. The two algorithms are evaluated on a dataset of image sequences of cotton plants captured in a high throughput plant phenotyping platform. The algorithms may be generalized to any plant species to study the effect of abiotic stresses on sustainable agriculture practices.

Keywords: deep neural networks; dynamic time warping; image sequence analysis; spectral band difference segmentation; stress prediction; temporal stress propagation; time series modeling.

Grants and funding

This research is partially supported by the Nebraska Agricultural Experiment Station with funding from the Hatch Act capacity program (Accession Number 1011130) from the USDA National Institute of Food and Agriculture.