Extensive time and financial resources have been dedicated to address nonpoint sources of nitrogen and phosphorus in watersheds. Despite these efforts, many watersheds have not seen substantial improvement in water quality. The objective of this study is to review the literature and investigate key factors affecting the lack of improvement in nutrient levels in waterways in urban and agricultural regions. From 94 studies identified in the academic literature, we found that, although 60% of studies found improvements in water quality after implementation of Best Management Practices (BMPs) within the watershed, these studies were mostly modeling studies rather than field monitoring studies. For studies that were unable to find improvements in water quality after the implementation of BMPs, the lack of improvement was attributed to lack of knowledge about BMP functioning, lag times, nonoptimal placement and distribution of BMPs in the watershed, postimplementation BMP failure, and socio-political and economic challenges. We refer to these limiting factors as known unknowns. We also acknowledge the existence of unknown unknowns that hinder further improvement in BMP effectiveness and suggest that machine learning, approaches from the field of business and operations management, and long-term convergent studies could be used to resolve these unknown unknowns.