All-pass phase shifting (APS), which involves a wave propagating at a constant, unitary amplitude but with pure phase variation, is extremely desired in many optoelectronic applications. In this work, we propose a method of realizing APS by out-of-plane excitation of a topologically enabled unidirectional guided resonance (UGR), which resides in a photonic crystal slab with P or C2z symmetries. Briefly, the symmetries and unidirectional features reduce the number of ports to one that simultaneously adds or drops energy. As a result, the phase independently shifts by varying the frequency but the amplitude remains as unitary under plane wave excitation. Theory and simulations confirm our findings. A paradox that the background contribution deviates from Fabry-Perot resonance is clarified from a multi-resonances picture.