Imaging-based measurement methods of polarization aberration (PA) are indispensable in hyper-numerical aperture projection optics for advanced lithography. However, the current methods are derived from the Kirchhoff model and ignore the 3D mask effect of the test mask, which will impact the measurement accuracy. In this paper, a novel imaging-based measurement method of PA is proposed based on a rigorous imaging model to improve the measurement accuracy. Through the quantitative description of the 3D mask effect, a rigorous imaging-based measurement model of PA is established. A synchronous orientation measurement method is designed to effectively reduce the cost of establishing the overdetermined equations. A deep neural network is used to retrieve the PA accurately. Simulations show that the proposed method effectively eliminates the impact of the 3D mask effect of test mask on PA measurement, and the measurement error is reduced by 72% compared with the measurement method based on the Kirchhoff model.