A solution to the problem of disturbing effect of the background Faraday rotation in the cryostat windows on longitudinal magneto-optical Kerr effect (LMOKE) measured under vacuum conditions and/or at low temperatures is proposed. The method for eliminating the influence of Faraday rotation in cryostat windows is based on special arrangement of additional mirrors placed on sample holder. In this arrangement, the orientation of the cryostat window is perpendicular to the light beam direction and parallel to an external magnetic field generated by the H-frame electromagnet. The operation of the LMOKE magnetometer with the special sample holder based on polarization modulation technique with a photo-elastic modulator is theoretically analyzed with the use of Jones matrices, and formulas for evaluating of the actual Kerr rotation and ellipticity of the sample are derived. The feasibility of the method and good performance of the magnetometer is experimentally demonstrated for the LMOKE effect measured in Fe/Au multilayer structures. The influence of imperfect alignment of the magnetometer setup on the Kerr angles, as derived theoretically through the analytic model and verified experimentally, is examined and discussed.