Hierarchically structured phononic crystals are proposed for filtering multiple frequency bands. The advantages of using structural hierarchy come from its multiscale periodicity and the increased number of controllable parameters, which contribute to open multiple bandgaps in broadband frequency ranges and adjust the positions of those bandgaps. By deriving a transfer-matrix-based theoretical formula, hierarchical phononic crystals are designed that filter the frequency bands for randomly selected frequencies in the ultrasonic range of 20 kHz to 10 MHz. Their wave-filtering capability is demonstrated by using numerical simulations with consideration of material loss. By comparing the transmittance spectra of the hierarchical phononic crystals with those of conventional ones, the structural hierarchy of the former is shown to be advantageous in filtering multiple frequency bands.