Unusual features of the vibrational density of states D(ω) of glasses allow one to rationalize their peculiar low-temperature properties. Simulational studies of D(ω) have been restricted to studying poorly annealed glasses that may not be relevant to experiments. Here we report on D(ω) of zero-temperature glasses with kinetic stabilities ranging from poorly annealed to ultrastable glasses. For all preparations, the low-frequency part of D(ω) splits between extended and quasi-localized modes. Extended modes exhibit a boson peak crossing over to Debye behavior (Dex(ω) ~ ω2) at low-frequency, with a strong correlation between the two regimes. Quasi-localized modes obey Dloc(ω) ~ ω4, irrespective of the stability. The prefactor of this quartic law decreases with increasing stability, and the corresponding modes become more localized and sparser. Our work is the first numerical observation of quasi-localized modes in a regime relevant to experiments, and it establishes a direct connection between glasses' stability and their soft vibrational modes.