X-ray imaging by means of a grating-based Talbot-Lau interferometer has become an important tool for a wide variety of application areas such as security, medical and materials analysis. Imaging modalities include attenuation, differential phase contrast, and visibility contrast (or so-called dark field). We have developed a novel modeling approach based on ray tracing with commercially available software (Zemax OpticStudio) that yields image projections for all three modalities. The results compare favorably with experimental findings. Our polychromatic ray-based model accommodates realistic 3-D CAD objects with tailored materials properties and also allows for both surface and bulk scattering. As such, the model can simulate imaging of complicated objects as well as assist in a physical understanding of experimental projection details.