In this paper, the annular folded lens (AFL) is applied to the realization of a miniaturized system for the visible and near-IR spectrums (0.45-1.1μm). In order to correct the chromatic aberration, a hybrid AFL is designed with the multilayer diffractive optical element (MLDOE) in which the substrate materials are precision molded glasses. We propose a new design method of the MLDOE to improve the polychromatic integral diffraction efficiency (PIDE) that makes it suitable for the optical path of the AFL. By comparing the characteristic angle weighted PIDE (CAW-PIDE), the optimal microstructure heights of the MLDOE can be obtained, and the effect of diffraction efficiency on image quality can be minimized for the entire incident angle range. The design results show that the ratio of total length to the focal length is only 0.332, and comprehensive modulation transfer function considering the diffraction efficiency is larger than 0.26 at 166 lp/mm. This study can provide a new idea for designing a broadband, miniaturized, and low-cost imaging system.