Shear stress is an important index to evaluate the rheological behavior of magnetorheological fluids (MRFs), which is not only related to the properties of ferromagnetic particles, but also the viscosity of the carrier. However, the research related to the carrier viscosity is quite lacking, and the mechanism of its effect on shear stress is still unclear. In this work, the carrier viscosity effect on the microstructure of MRFs under shearing was investigated via numerical simulations, and the relationship between chain inclination and carrier viscosity was presented for the first time. It was found that the deflection angle of the chain increases with the increase of carrier viscosity. Based on the simulation results, the relationship between the shear resistance induced by the magnetic field and the deflection angle of the chain was studied. Finally, a constitutive model incorporating the mechanism of the viscosity effect on shear stress was proposed, and the calculated results agreed well with the experimental data. This work provides new insights into the effect of carrier viscosity and can help us to better understand the corresponding microscopic mechanism.