In this report, the strong-dependence of low-frequency (terahertz) vibrational dynamics on weak and long-range forces in crystals is leveraged to determine the bulk magnetic configuration of iron phosphate - a promising material for cathodes in lithium ion batteries. We demonstrate that terahertz time-domain spectroscopy - coupled with quantum mechanical simulations - can discern between various spin configurations in FePO4. Furthermore, the results of this work unambiguously show that the well-accepted space group symmetry for FePO4 is incorrect, and the low-frequency spectroscopic measurements provide a clearer picture of the correct structure over the gold-standard of X-ray diffraction. This work opens the door for characterizing, predicting, and interpreting crystalline magnetic ordering using low-frequency vibrational spectroscopy.