Achieving thermodynamic faithfulness and transferability across state points is an outstanding challenge in the bottom-up coarse graining of molecular models, with many efforts focusing on augmenting the form of coarse-grained interaction potentials to improve transferability. Here, we revisit the critical role of the simulation ensemble and the possibility that even simple models can be made more predictive through a smarter choice of ensemble. We highlight the efficacy of coarse graining from ensembles where variables conjugate to the thermodynamic quantities of interest are forced to respond to applied perturbations. For example, to learn activity coefficients, it is natural to coarse grain from ensembles with spatially varying external potentials applied to one species to force local composition variations and fluctuations. We apply this strategy to coarse grain both an atomistic model of water and methanol and a binary mixture of spheres interacting via Gaussian repulsions and demonstrate near-quantitative capture of activity coefficients across the whole composition range. Furthermore, the approach is able to do so without explicitly measuring and targeting activity coefficients during the coarse graining process; activity coefficients are only computed after-the-fact to assess accuracy. We hypothesize that ensembles with applied thermodynamic potentials are more "thermodynamically informative." We quantify this notion of informativeness using the Fisher information metric, which enables the systematic design of optimal bias potentials that promote the learning of thermodynamically faithful models. The Fisher information is related to variances of structural variables, highlighting the physical basis underlying the Fisher information's utility in improving coarse-grained models.