For a rough diffuser that fully depolarizes a linearly polarized laser, the speckle contrast of a fully developed speckle image can be reduced from 1 to 0.5 by polarization diversity. This reduction is achieved by rotating an x-polarized laser to the y-polarized orientation to form four independent speckle patterns with equal intensities during the detector exposure time. For the case of arbitrary rotation of the polarization, we derived a generalized speckle contrast formula for the superposed speckle patterns. This formula completes the theory of speckle contrast for the sum of correlated speckle intensities as it relates to polarization diversity.