Effective single particle potentials governing the motion of O2 and N3- anions have been determined by single crystal neutron diffraction at high temperatures for three samples of ZrO2 doped with different amounts of Y and N. Diffusion jumps take place directly to vacant nearest neighbour anion sites through the edges of the surrounding cation tetrahedra along (100)-directions. Activation enthalpies of migration for O (1.09 eV) and N (1.99 eV) are in good agreement with values obtained from tracer diffusion measurements (M. Kilo, C. Argirusis, G. Borchardt and R. A. Jackson, Phys. Chem. Chem. Phys., 2003, 5, 2219 and M. Kilo, M. A. Taylor, C. Argirusis, G. Borchardt, M. Lerch, O. Kaitasov and B. Lesage, Phys. Chem. Chem. Phys., 2004, 6, 3645). The diffusion process is facilitated by local short range order and anharmonic thermal vibrations. It is therefore advocated that the interactions with the phonons have to be taken into account in the description of the diffusion process.