Phosphoproteomics deals with the identification and quantification of thousands of phosphopeptides. Localizing the phosphorylation site is however much more difficult than establishing the identity of a phosphorylated peptide. Further, recent findings have raised doubts of the validity of the site assignments in large-scale phosphoproteomics data sets. To improve methods for site localization, we made use of a synthetic phosphopeptide library and SILAC-labeled peptides from whole cell lysates and analyzed these with high-resolution tandem mass spectrometry on an LTQ Orbitrap Velos. We validated gas-phase phosphate rearrangement reactions during collision-induced dissociation (CID) and used these spectra to devise a quantitative filter that by comparing signal intensities of putative phosphorylated fragment ions with their nonphosphorylated counterparts allowed us to accurately pinpoint which fragment ions contain a phosphorylated residue and which ones do not. We also evaluated higher-energy collisional dissociation (HCD) and found this to be an accurate method for correct phosphorylation site localization with no gas-phase rearrangements observed above noise level. Analyzing a large set of HCD spectra of SILAC-labeled phosphopeptides, we identified a novel fragmentation mechanism that generates a phosphorylation site-specific neutral loss derived x-ion, which directly pinpoints the phosphorylated residue. Together, these findings significantly improve phosphorylation site localization confidence.