Van Gogh Yellow refers to a family of lead chromate pigments widely used in the 19th century and often mixed with lead sulfate to obtain different yellow hues. Unfortunately, some paintings, such as the famous Sunflowers series, suffered degradation problems due to photoactivated darkening of once bright yellow areas, especially when irradiated with UV light. Recent advanced spectroscopic analyses have proved that this process occurs mostly where the pigment presents a sulfur-rich orthorhombic phase of a PbCr(1-x)S x O4 solid solution, while chromium-rich monoclinic phases are lightfast. However, the question of whether degradation is purely a surface phenomenon or if the bulk properties of sulfur-rich pigments trigger the process is still open. Here, we use first-principles calculations to unveil the role of sulfur in determining important bulk features such as structure, stability, and optical properties. From our findings, we suggest that degradation occurs via an initial local segregation of lead sulfate that absorbs at UV light wavelengths and provides the necessary energy for subsequent reduction of chromate ions into the greenish chromic oxide detected in experiments. In perspective, our results set reliable scientific foundations for further studies on surface browning phenomena and can help to chose the best strategy for the proper conservation of art masterpieces.