Speckle patterns observed in coherent optical imaging reflect important characteristic information of the scattering object. To capture speckle patterns, angular resolved or oblique illumination geometries are usually employed in combination with Rayleigh statistical models. We present a portable and handheld 2-channel polarization-sensitive imaging instrument to directly resolve terahertz (THz) speckle fields in a collocated telecentric back-scattering geometry. The polarization state of the THz light is measured using two orthogonal photoconductive antennas and can be presented in the form of the Stokes vectors of the THz beam upon interaction with the sample. We report on the validation of the method in surface scattering from gold-coated sandpapers, demonstrating a strong dependence of the polarization state on the surface roughness and the frequency of the broadband THz illumination. We also demonstrate non-Rayleigh first-order and second-order statistical parameters, such as degree of polarization uniformity (DOPU) and phase difference, for quantifying the randomness of polarization. This technique provides a fast method for broadband THz polarimetric measurement in the field and has the potential for detecting light depolarization in applications ranging from biomedical imaging to non-destructive testing.