Inositol hexakisphosphate (IP6) is an assembly cofactor for HIV-1. We report here that IP6 is also used for assembly of Rous sarcoma virus (RSV), a retrovirus from a different genus. IP6 is ~100-fold more potent at promoting RSV mature capsid protein (CA) assembly than observed for HIV-1 and removal of IP6 in cells reduces infectivity by 100-fold. Here, visualized by cryo-electron tomography and subtomogram averaging, mature capsid-like particles show an IP6-like density in the CA hexamer, coordinated by rings of six lysines and six arginines. Phosphate and IP6 have opposing effects on CA in vitro assembly, inducing formation of T = 1 icosahedrons and tubes, respectively, implying that phosphate promotes pentamer and IP6 hexamer formation. Subtomogram averaging and classification optimized for analysis of pleomorphic retrovirus particles reveal that the heterogeneity of mature RSV CA polyhedrons results from an unexpected, intrinsic CA hexamer flexibility. In contrast, the CA pentamer forms rigid units organizing the local architecture. These different features of hexamers and pentamers determine the structural mechanism to form CA polyhedrons of variable shape in mature RSV particles.