Coarse-grained (CG) models provide a computationally efficient method for rapidly investigating the long time- and length-scale processes that play a critical role in many important biological and soft matter processes. Recently, Izvekov and Voth introduced a new multiscale coarse-graining (MS-CG) method [J. Phys. Chem. B 109, 2469 (2005); J. Chem. Phys. 123, 134105 (2005)] for determining the effective interactions between CG sites using information from simulations of atomically detailed models. The present work develops a formal statistical mechanical framework for the MS-CG method and demonstrates that the variational principle underlying the method may, in principle, be employed to determine the many-body potential of mean force (PMF) that governs the equilibrium distribution of positions of the CG sites for the MS-CG models. A CG model that employs such a PMF as a "potential energy function" will generate an equilibrium probability distribution of CG sites that is consistent with the atomically detailed model from which the PMF is derived. Consequently, the MS-CG method provides a formal multiscale bridge rigorously connecting the equilibrium ensembles generated with atomistic and CG models. The variational principle also suggests a class of practical algorithms for calculating approximations to this many-body PMF that are optimal. These algorithms use computer simulation data from the atomically detailed model. Finally, important generalizations of the MS-CG method are introduced for treating systems with rigid intramolecular constraints and for developing CG models whose equilibrium momentum distribution is consistent with that of an atomically detailed model.