Hydration free energy estimation of small molecules from all-atom simulations was widely investigated in recent years, as it provides an essential test of molecular force fields and our understanding of solvation effects. While explicit solvent representations result in highly accurate models, they also require extensive sampling due to the high number of solvent degrees of freedom. Implicit solvent models, such as those based on the generalized Born model for electrostatic solvation effects and a solvent accessible surface area term for nonpolar contributions (GBSA), significantly reduce the number of degrees of freedom and the computational cost to estimate hydration free energies. However, a recent survey revealed a gap in the accuracy between explicit TIP3P solvent estimates and those computed with many common GBSA models. Here we address this shortcoming by providing a thorough comparison of the performance of three implicit solvent models with different nonpolar contributions and a generalized Born term to estimate experimental hydration free energies. Starting with a minimal set of only ten atom types, we demonstrate that a nonpolar term with atom type dependent surface tension coefficients in combination with an accurate generalized Born term and fully optimized parameters performs best in estimating hydration free energies, even yielding comparable results to the explicit TIP3P water model. Analysis of our results provides evidence that the asymmetric behavior of water around oppositely charged atoms is one of the main sources of error for two of the three implicit solvent models. Explicitly accounting for this effect in the parameterization reduces the corresponding errors, suggesting this as a general strategy for improving implicit solvent models. The findings presented here will help to improve the existing generalized Born based implicit solvent models implemented in state-of-the-art molecular simulation packages.