The fallacy in the use of the "best-fit" solution in hydrologic modeling

Abstract

Using the parameters associated with the best-fit simulation (i.e., the simulation with the highest objective function value) to represent a calibrated hydrological model is inadequate. The reason is that the calibrated models best objective function value is usually not significantly different from the next best value or the values after that. This non-uniqueness of the objective function values causes a problem because the best solution's parameters are often significantly different from the next best set of parameters. Therefore, only using the best simulation parameters as the calibrated model's sole parameters to interpret the watershed processes or perform further modeling analyses could produce misleading results. Furthermore, the lack of pristine watersheds makes the task of watershed-scale calibration increasingly challenging. Subjective thresholds of acceptable performance criteria suggested by some researchers, based on comparing the measured and the best solution signals, are often not achievable. Hence, to obtain a satisfactory fit, researchers and practitioners are often forced to compromise the science behind their work. This article discusses the fallacy in using the best-fit solution in hydrologic modeling. A two-factor statistic to assess the goodness of calibration/validation is discussed, considering model output uncertainty.

Keywords: Calibration; Deterministic calibration; Model uncertainty; Performance criteria; Stochastic calibration.