Utilizing the polarization analysis in underwater imaging can effectively suppress the scattered light and help to restore target signals in turbid water. Neural network-based solutions can also boost the performance of polarimetric underwater imaging, while most of the existing networks are pure data driven which suffer from ignoring the physical mode. In this paper, we proposed an effective solution that informed the polarimetric physical model and constrains into the well-designed deep neural network. Especially compared with the conventional underwater imaging model, we mathematically transformed the two polarization-dependent parameters to a single parameter, making it easier for the network to converge to a better level. In addition, a polarization perceptual loss is designed and applied to the network to make full use of polarization information on the feature level rather than on the pixel level. Accordingly, the network was able to learn the polarization modulated parameter and to obtain clear de-scattered images. The experimental results verified that the combination of polarization model and neural network was beneficial to improve the image quality and outperformed other existing methods, even in a high turbidity condition.