Relaxor ferroelectrics, which can exhibit exceptional electromechanical coupling, are some of the most important functional materials, with applications ranging from ultrasound imaging to actuators. Since their discovery, their complex nanoscale chemical and structural heterogeneity has made the origins of their electromechanical properties extremely difficult to understand. Here, we employ aberration-corrected scanning transmission electron microscopy to quantify various types of nanoscale heterogeneities and their connection to local polarization in the prototypical relaxor ferroelectric system Pb(Mg1/3Nb2/3)O3-PbTiO3. We identify three main contributions that each depend on Ti content: chemical order, oxygen octahedral tilt and oxygen octahedral distortion. These heterogeneities are found to be spatially correlated with low-angle polar domain walls, indicating their role in disrupting long-range polarization and leading to nanoscale domain formation and the relaxor response. We further locate nanoscale regions of monoclinic-like distortion that correlate directly with Ti content and electromechanical performance. Through this approach, the connections between chemical heterogeneity, structural heterogeneity and local polarization are revealed, validating models that are needed to develop the next generation of relaxor ferroelectrics.