Because of the low calibration precision caused by reflector distortion in catadioptric vision systems with conic mirrors, we proposed a double distortion correction model based on the panoramic image rectification. The lens distortion and the reflector distortion are analyzed independently in the mathematic model of the double distortion correction. Moreover, the modified model in this paper added the trigonometric functions into the polynomial model to solve the combined distortion. Additionally, extensive experiments, both simulative and real data, have been carried out and show that the proposed model achieves comparable measurement accuracy in contrast with the traditional distortion method.