A spectral confocal displacement and morphology measurement device with a photopolymer-based coaxial holographic lens as a high dispersion element is developed. The linear dependence of the axial spatial position on the peak wavelength of dispersion spectrum provides a high accuracy and large range for measuring displacement and morphology. In the linear dispersion region, accompanied with a 120 nm shift of peak wavelength, the measure position range exceeds 20 mm. The available experimental accuracy of displacement and morphology can reach 47.5 μm/0.5 nm using a commercial optical fiber spectrum with a resolution of 0.5 nm. Utilizing thin polymer-based holographic lens with high dispersion can effectively compact the device size. Simultaneously, it can provide a large axial dispersion for measuring the spatial position characterization compared with the traditional glass-based dispersion lens group. A holographic optical lens based on a photopolymer is expected to apply in high-precision surface morphology measurement of large-scale macroscopic objects. It will improve the measurement accuracy and accelerate the development of holographic optical elements.