X-ray phase contrast imaging (XPCI) provides enhanced image contrast beyond absorption-based x-ray imaging alone due to refraction and diffraction from gradients in the object material density. It is sensitive to small variations in density, such as internal voids, cracks, grains, defects, and material flow, as well as to stronger density variations such as from a shock wave. Beyond its initial use in biology and materials science, XPCI is now routinely used in inertial confinement fusion (ICF) and high energy density (HED) research, first to characterize ICF capsules and targets, and later applied in dynamic experiments, where coherent x-ray sources, ultrafast x-ray pulses, and high temporal and spatial resolution are required. In this Review article, XPCI image formation theory is presented, its diverse use in ICF and HED research is discussed, the unique requirements for ultrafast XPCI imaging are given, as well as current challenges and issues in its use.