The existence of nearby obstruction causes significant errors in depth sensing for time-of-flight cameras, namely multipath interference. A polarized time-of-flight system is established for multipath interference mitigation. Based on polarization cues and the phasor representation of time-of-flight imaging, the proposed method acquires depth maps in high accuracy when specular dominant obstruction is in path. Both rough and smooth targets are applicable in our approach even though they have distinct polarization characteristics. Several experiments with different types of targets and various obstructions confirm the effectiveness of our method qualitatively and quantitatively.