Owing to high accuracy in the whole measurement range, the compensator-rotating method is a main approach for the single-point ellipsometric measurement. The disadvantage of this method is the complexity resulting from the retardation calibration for the rotating compensator. The compensator-rotating imaging ellipsometer, a system combining the optical microscope and the single-point measurement ellipsometer, faces a similar issue. An important problem for the imaging ellipsometer is that the retardation of the compensator manifests inhomogeneity over the view field. Here, we propose a calibration method of the retardation of the compensator for the imaging ellipsometer. The approach was tested experimentally with a homebuilt imaging ellipsometer by generating maps of the ellipsometric parameters $\Delta$Δ and $\psi$ψ of samples of a Si wafer and an opaque Cr thin film.