Data pipeline for managing field experiments

Jian Liu; Rogerio Cichota; Stephanie Langer; Eric Burgueño; Alexandre Michel

doi:10.1016/j.mex.2023.102031

Data pipeline for managing field experiments

MethodsX. 2023 Jan 19:10:102031. doi: 10.1016/j.mex.2023.102031. eCollection 2023.

Authors

Jian Liu¹, Rogerio Cichota¹, Stephanie Langer¹, Eric Burgueño¹, Alexandre Michel¹

Affiliation

¹ The New Zealand Institute for Plant and Food Research Limited, New Zealand.

Abstract

As agricultural and environmental research projects become more complex, increasingly with multiple outcomes, the demand for technical support with experiment management and data handling has also increased. Interactive visualisation solutions are user-friendly and provide direct information to facilitate decision making with timely data interpretation. Existing off-the-shelf tools can be expensive and require a specialist to conduct the development of visualisation solutions. We used open-source software to develop a customised near real-time interactive dashboard system to support science experiment decision making. Our customisation allowed for: • Digitalised domain knowledge via open-source solutions to develop decision support systems. • Automated workflow that only executed the necessary components. • Modularised solutions for low maintenance cost and upgrades.

Keywords: Data pipeline for managing field experiments; Data streamlining; Experiment management; Real-time visualisation.