Coarse-grained models are becoming a competitive alternative for modeling processes that occur over time and length scales beyond the reach of fully atomistic molecular simulations. Ideally, coarse-grained models should not only achieve high computational efficiency but also provide accurate predictions and fundamental insight into the role of molecular interactions, the characteristic behavior, and properties of the system they model. In this work we derive a series of monatomic coarse-grained water models mX(REM) from the most popular atomistic water models X = TIP3P, SPC/E, TIP4P-Ew, and TIP4P/2005, using the relative entropy minimization (REM) method. Each coarse-grained water molecule is represented by a single particle that interacts through short-ranged anisotropic interactions that encourage the formation of "hydrogen-bonded" structures. We systematically investigate the features of the coarse-grained models in reproducing over 20 structural, dynamic, and thermodynamic properties of the reference atomistic water models-including the existence and locus of the characteristic density anomaly. The mX(REM) coarse-grained models reproduce quite faithfully the radial and angular distribution function of water, produce a temperature of maximum density (TMD), and stabilize the ice I crystal. Moreover, the ratio between the TMD and the melting temperature of the crystal in the mX(REM) models and liquid-ice equilibrium properties show reasonable agreement with the results of the corresponding atomistic models. The mX(REM) models, however, severely underestimate the cohesive energy of the condensed water phases. We investigate which specific limitations of the coarse-grained models arise from the REM methodology, from the monatomic nature of the models, and from the Stillinger-Weber interaction potential form. Our analysis indicates that a small compromise in the accuracy of structural properties can result in a significant increase of the overall accuracy and representability of the coarse-grained water models. We evaluate the accuracy of the atomistic and the monatomic anisotropic coarse-grained water models, including the mW water model, in reproducing experimental water properties. We find that mW and mTIP4P/2005(REM) score closer to experiment than widely used atomistic water models. We conclude that monatomic models of water with short-range, anisotropic "hydrogen-bonding" three-body interactions can be competitive in accuracy with fully atomistic models for the study of a wide range of properties and phenomena at less than 1/100th of the computational cost.