There is renewed interest in the structure of the essential amino acid phenylalanine in the solid state. Three new polymorphs were found in the years 2012 to 2014. Here, we investigate the structure, stability, and energetical ordering of these phases using first-principles simulations at the level of density functional theory incorporating van der Waals interactions. Two of the distinct crystal forms are found to be structurally similar and energetically very close after vibrational free energy corrections have been taken into account. Infrared absorption spectra are likewise calculated and compared to experimental measurements. By combining measurements obtained with a commercial Fourier transform infra-red spectrometer and a homemade air-photonics-based THz time domain spectrometer, we could carry out this comparison in the vibrational frequency region from 1 to 40 THz. The excellent agreement of the line positions and the established energy ranking allow us to identify the most stable polymorph of phenylalanine.