Despite its simple crystal structure, layered boron nitride features a surprisingly complex variety of phonon-assisted luminescence peaks. We present a combined experimental and theoretical study on ultraviolet-light emission in hexagonal and rhombohedral bulk boron nitride crystals. Emission spectra of high-quality samples are measured via cathodoluminescence spectroscopy, displaying characteristic differences between the two polytypes. These differences are explained using a fully first-principles computational technique that takes into account radiative emission from "indirect," finite-momentum excitons via coupling to finite-momentum phonons. We show that the differences in peak positions, number of peaks, and relative intensities can be qualitatively and quantitatively explained, once a full integration over all relevant momenta of excitons and phonons is performed.