The computational and conceptual simplifications realized by coarse-grain (CG) models make them a ubiquitous tool in the current computational modeling landscape. Building block based CG models, such as the Martini model, possess the key advantage of allowing for a broad range of applications without the need to reparametrize the force field each time. However, there are certain inherent limitations to this approach, which we investigate in detail in this work. We first study the consequences of the absence of specific cross Lennard-Jones parameters between different particle sizes. We show that this lack may lead to artificially high free energy barriers in dimerization profiles. We then look at the effect of deviating too far from the standard bonded parameters, both in terms of solute partitioning behavior and solvent properties. Moreover, we show that too weak bonded force constants entail the risk of artificially inducing clustering, which has to be taken into account when designing elastic network models for proteins. These results have implications for the current use of the Martini CG model and provide clear directions for the reparametrization of the Martini model. Moreover, our findings are generally relevant for the parametrization of any other building block based force field.