Investigating the applicability and assumptions of the regression relationship between flow discharge and nitrogen concentrations for load estimation

Jung-Hun Song; Younggu Her; Youn Shik Park; Kwangsik Yoon; Hakkwan Kim

doi:10.1016/j.heliyon.2023.e23603

Investigating the applicability and assumptions of the regression relationship between flow discharge and nitrogen concentrations for load estimation

Heliyon. 2023 Dec 19;10(1):e23603. doi: 10.1016/j.heliyon.2023.e23603. eCollection 2024 Jan 15.

Authors

Jung-Hun Song^{1

2

3}, Younggu Her³, Youn Shik Park⁴, Kwangsik Yoon⁵, Hakkwan Kim⁶

Affiliations

¹ Institute of Green Bio Science and Technology, Seoul National University, Pyeongchang 25354, Republic of Korea.
² Department of Integrated Major in Global Smart Farm, Seoul National University, Seoul 08826, Republic of Korea.
³ Department of Agricultural and Biological Engineering & Tropical Research and Education Center, University of Florida, Homestead, FL 33031, USA.
⁴ Department of Rural Construction Engineering, Kongju National University, Yesan 32439, Republic of Korea.
⁵ Department of Rural and Biosystems Engineering & Education and Research Unit for Climate-Smart Reclaimed Tideland Agriculture (BK21 four), Chonnam National University, Gwangju 61186, Republic of Korea.
⁶ Graduate School of International Agricultural Technology and Institutes of Green Bio Science and Technology, Seoul National University, Pyeongchang 25354, Republic of Korea.

Abstract

The regression relationship between water discharge rates and nutrient concentrations can provide a quick and straightforward way to estimate nutrient loads. However, recent studies indicated that the relationship might produce large biases in load estimates and, therefore, may not be applicable in certain types of cases. The goal of this study is to explore the theoretical reasons behind the selective applicability of the regression relationship between flow rates and nitrate + nitrite concentrations. For this study, we examined daily flow and nitrate + nitrite concentration observations made at the outlets of 22 watersheds monitored by the Heidelberg Tributary Loading Program (HTLP). The statistical relationship between the flow rates and concentrations was explored using regression equations offered by the LOAD ESTimator (LOADEST). Results demonstrated that the use of the regression equations provided nitrate + nitrite load estimates at acceptable accuracy levels ( $N S E \geq 0.35$ and $| P B I A S | \leq 30.0$ %) in 14 watersheds (64 % of 22 study watersheds). The regression relationships provided highly biased results at eight watersheds (36 %), implying their limited applicability. The heteroscedasticity of the residuals led to the high bias and resulting inaccurate regression, which was commonly found in watersheds where low flow had high nitrate + nitrite concentration variations. Conversely, the regression relationships provided acceptable accuracy for watersheds that had a relatively constant variance of the nitrate + nitrite concentrations. The results indicate that the homoscedasticity of residuals is the key assumption to be satisfied to estimate nitrate + nitrite loads from a statistical regression between flow discharge and nitrate + nitrite concentrations. The transport capacity (capacity-limited) concept implicitly assumed in the regression relationship between flow discharge and nitrate + nitrite concentrations is not always applicable, especially to agricultural areas in which nitrate + nitrite loads are highly variable depending on management practices (supply-limited). The findings suggest that the regression relationship should be carefully applied to areas in which intensive agricultural activities, including crop management and conservation practices, are implemented. Thus, the transport capacity concept is reasonably regarded to contribute to the homoscedasticity of residuals.

Keywords: Flow discharge; LOADEST; Regression model; Supply-limited; Transport-limited; nitrate+nitrite.