Visible light Zernike phase contrast (ZPC) microscopy is a well established method for imaging weakly absorbing samples. The method is also used with hard x-ray photon energies for structural evaluation of material science and biological applications. However, the method suffers from artifacts that are inherent for the Zernike image formation. In this Letter, we investigate their origin and experimentally show how to suppress them in x-ray full-field ZPC microscopy based on diffractive x-ray optics.