We report that the interference pattern of Young's double-slit experiment changes as a function of polarization in the sub-wavelength diffraction regime. Experiments carried out with terahertz time-domain spectroscopy reveal that diffracted waves from sub-wavelength-scale slits exhibit either positive or negative phase shift with respect to Gouy phase depending on the polarization. Theoretical explanation based on the induction of electric current and magnetic dipole in the vicinity of the slits shows an excellent agreement with the experimental results.