Lithium phosphorus oxynitride, also known as Lipon, solid-state electrolytes are at the center of the search for solid-state Li metal batteries. Key to the performance of Lipon is a combination of high Li content, amorphous character, and the incorporation of N into the structure. Despite the material's importance, our work presents the first study to fully resolve the structure of Lipon using a combination of ab initio molecular dynamics, density functional theory, neutron scattering, and infrared spectroscopy. The modeled and experimental results have exceptional agreement in both neutron pair distribution function and infrared spectroscopy. Building on this synergy, the structural models show that N forms both bridges between two phosphate units and nonbridging apical N. We further show that as the Li content is increased the ratio of bridging to apical N shifts from being predominantly bridging at Li contents around 2.5:1 Li:P to only apical N at higher Li contents of 3.38:1 Li:P. This crossover from bridging to apical N appears to directly correlate with and explain both the increase in ionic conductivity with the incorporation of N and the ionic conductivity trends found in the literature.