For a particle travelling through an interferometer, the trade-off between the available which-way information and the interference visibility provides a lucid manifestation of the quantum mechanical wave-particle duality. Here we analyse this relation for a particle possessing an internal degree of freedom such as spin. We quantify the trade-off with a general inequality that paints an unexpectedly intricate picture of wave-particle duality when internal states are involved. Strikingly, in some instances which-way information becomes erased by introducing classical uncertainty in the internal degree of freedom. Furthermore, even imperfect interference visibility measured for a suitable set of spin preparations can be sufficient to infer absence of which-way information. General results are illustrated with a proof-of-principle single-photon experiment.