This work reports on the influence of phosphorous atoms on the phase separation process and optical properties of silicon nanocrystals (Si-NCs) embedded in phosphorus doped SiO/SiO2 multilayers. Doped SiO/SiO2 multilayers with different P contents have been prepared by co-evaporation and subsequently annealed at different temperatures up to 1100 °C. The sample structure and the localization of P atoms were both studied at the nanoscale by scanning transmission electron microscopy and atom probe tomography. It is found that P incorporation modifies the mechanism of Si-NC growth by promoting the phase separation during the post-growth-annealing step, leading to nanocrystal formation at lower annealing temperatures as compared to undoped Si-NCs. Hence, the maximum of Si-NC related photoluminescence (PL) intensity is achieved for annealing temperatures lower than 900 °C. It is also demonstrated that the Si-NCs mean size increases in the presence of P, which is accompanied by a redshift of the Si-NC related emission. The influence of the phosphorus content on the PL properties is studied using both room temperature and low temperature measurements. It is shown that for a P content lower than about 0.1 at%, P atoms contribute to significantly improve the PL intensity. This effect is attributed to the P-induced-reduction of the number of non-radiative defects at the interface between Si-NCs and SiO2 matrix, which is discussed in comparison with hydrogen passivation of Si-NCs. In contrast, for increasing P contents, the PL intensity strongly decreases, which is explained by the growth of Si-NCs reaching sizes that are too large to ensure quantum confinement and to the localization of P atoms inside Si-NCs.