Stray light in an optical system is unwanted parasitic light that may degrade performance. It can originate from different sources and may lead to different problems in the optical system such as fogging, ghost images for imagers, or inaccurate measurements for time of flight applications. One of the root causes is the reflectivity of the sensor itself. In this paper we present a new optical simulation methodology to analyze the stray light contribution due to the sensor reflectivity by coupling electromagnetic simulation (to calculate the pixels' bidirectional reflectance distribution function, also named BRDF) and ray-tracing simulation (for stray light analysis of the camera module). With this simulation flow we have been able to reproduce qualitatively red ghost images observed on different sensors in our laboratory.