The linear and circular polarization ratios for clusters of spherical particles averaged over multiple orientations show a systematic pattern as a function of the refractive index and the size parameter. We show that, at backscattering, the depolarizing behavior of orientation-averaged clusters of spheres can be approximated by second-order scattering of bispheres. The pattern is relatively invariable in terms of the number of particles. We also demonstrate the significance of the near-field effects for polarization at backscattering.