While terahertz time domain spectroscopy (THz-TDS) is a well-established technique, polarization sensitive measurements are challenging due to the need of broadband polarization devices. Here, we characterize our recently introduced multi-contact photoconductive detector antenna with a response matrix analysis. We show that the lead lines attached to electrodes reduce the antenna symmetry and thereby influence the properties of the response matrices. With a wire grid polarizer, we simulate a sample influencing the polarization angle and the intensity of the incident THz pulse. Evaluating the measurements with the response matrix analysis, our results show a well agreement of the adjusted and measured polarization angles and intensities over a frequency range from 0.25 to 0.8 THz.