Modern design of common adhesives, composites and polymeric parts makes use of polymer glasses that are stiff enough to maintain their shape under a high stress while still having a ductile behavior after the yield point. Typically, material compositions are tuned with co-monomers, polymer blends, plasticizers, or other additives to arrive at a tradeoff between the elastic modulus and toughness. In contrast, strong changes to the mechanics of a glass are possible by changing only the molecular packing during vitrification or even deep in the glassy state. For example, physical aging or processing techniques such as physical vapor deposition increase the density, embrittle the material, and increase elastic modulus. Here, we use molecular simulations, validated by positron annihilation lifetime spectroscopy (PALS) and quasi-elastic neutron scattering, to understand the free volume distribution and the resulting dynamics of glassy co-polymers where the composition is systemically varied between polar 5-norbornene-2-methanol (NBOH) and non-polar ethylidene norbornene (ENB) monomers. In these polymer glasses, we analyze the structural features of the unoccupied volume using clustering analysis, where the clustering is parameterized to reproduce experimental measurements of the same features from PALS. Further, we analyze the dynamics, quantified by the Debye-Waller factor, and compare the results with softer, lower density states. Our findings indicate that faster structural relaxations and potentially improved ductility are possible through changes to the geometric structure and fraction of the free volume, and that the resulting changes to the glass dynamics are comparable to large changes in the monomer composition.