Lasers have been regarded as the potential illumination source for next generation projectors. With the additional feature of coherency, diffractive optical elements (DOEs) become available in the illumination light path. DOEs in laser projection display systems add strong beam-shaping ability, good uniform performance and small size, which makes it is possible to realize efficient and uniform illumination on a spatial light modulator (SLM). Moreover, it is helpful for the simplification and compactness of illumination optics. This paper proposed what we believe is a novel RGB laser projection display illumination system based on a DOE, which used the DOE and optical path compensation system (OPCS) as the beam-shaping and relay system (BSRS), instead of a light pipe and relay lens group in the conventional laser projection display illumination system. We designed the DOE and established the simulation model of the illumination system. The simulation results show that the new illumination system is simple and compact, the illumination uniformity is more than 90%, the illumination efficiency of RGB illumination in BSRS is more than 75%, and the numerical aperture (NA) of the illumination beam is about 0.01.