The characterization and manipulation of polarization state at single photon level are of great importance in research fields such as quantum information processing and quantum key distribution, where photons are normally delivered using single mode optical fibers. To date, the demonstrated polarimetry measurement techniques based on a superconducting nanowire single photon detector (SNSPD) require the SNSPD to be either highly sensitive or highly insensitive to the photon's polarization state, therefore placing an unavoidable challenge on the SNSPD's design and fabrication processes. In this article, we present the development of an alternative polarimetry measurement technique, of which the stringent requirement on the SNSPD's polarization sensitivity is removed. We validate the proposed technique by a rigorous theoretical analysis and comparisons of the experimental results obtained using a fiber-coupled SNSPD with a polarization extinction ratio of ∼2 to that obtained using other well-established known methods. Based on the full Stokes data measured by the proposed technique, we also demonstrate that at the single photon level (∼ -100 dBm), the polarization state of the photon delivered to the superconducting nanowire facet plane can be controlled at will using a further developed algorithm. Note that other than the fiber-coupled SNSPD, the only component involved is a quarter-wave plate (no external polarizer is necessary), which when aligned well has a paid insertion loss less than 0.5 dB.