We validate an automated implementation of a combined Molecular Mechanics/Generalized Born Surface Area (MM/GBSA) method (VSGB 2.0 energy model) on a large and diverse selection of protein-ligand complexes (855 complexes). Although this data set is diverse with respect to both protein families and ligands, after carefully removing flawed structures, a significant correlation (R(2) = 0.63) between calculated and experimental binding affinities is obtained. Consistent explanations for "outlier" complexes are found. Visual analysis of the crystal structures and recourse to the original literature reveal that neglect of explicit solvent, ligand strain, and entropy contribute to the under- and overestimation of computed affinities. The limits of the Molecular Mechanics/Implicit Solvent approach to accurately estimate protein-ligand binding affinities is discussed as is the influence of the quality of protein-ligand complexes on computed free energy binding values.