We present a method with which to recover the intrinsic shading and reflectance characteristics of multicolored three-dimensional objects in a single image, with which realistic new scenes can be synthesized. A color watershed algorithm, which is based on a regularized dichromatic fitting error, is proposed for robust image segmentation. For shading recovery in small regions, a weighted interpolation is employed, whereas in large regions the reflectance and shading are calculated based on the assumption of gradual shape variation. It is demonstrated that the proposed method is promising and can be applied in image simulation.