We perform molecular dynamics simulations to investigate the effect of a glass preparation on its yielding transition under oscillatory shear. We use swap Monte Carlo to investigate a broad range of glass stabilities from poorly annealed to highly stable systems. We observe a qualitative change in the nature of yielding, which evolves from ductile to brittle as glass stability increases. Our results disentangle the relative role of mechanical and thermal annealing on the mechanical properties of amorphous solids, which is relevant for various experimental situations from the rheology of soft materials to fatigue failure in metallic glasses.