We present a method to parameterize heterogeneous elastic network models (heteroENMs) of proteins to reproduce the fluctuations observed in atomistic simulations. Because it is based on atomistic simulation, our method allows the development of elastic coarse-grained models of proteins under different conditions or in different environments. The method is simple and applicable to models at any level of coarse-graining. We validated the method in three systems. First, we computed the persistence length of ADP-bound F-actin, using a heteroENM model. The value of 6.1 +/- 1.6 microm is consistent with the experimentally measured value of 9.0 +/- 0.5 microm. We then compared our method to a uniform elastic network model and a realistic extension algorithm via covariance Hessian (REACH) model of carboxy myoglobin, and found that the heteroENM method more accurately predicted mean-square fluctuations of alpha-carbon atoms. Finally, we showed that the method captures critical differences in effective harmonic interactions for coarse-grained models of the N-terminal Bin/amphiphysin/Rvs (N-BAR) domain of amphiphysin, by building models of N-BAR both bound to a membrane and free in solution.