X-ray dark-field imaging enables a spatially-resolved visualization of ultra-small-angle X-ray scattering. Using phantom measurements, we demonstrate that a material's effective dark-field signal may be reduced by modification of the visibility spectrum by other dark-field-active objects in the beam. This is the dark-field equivalent of conventional beam-hardening, and is distinct from related, known effects, where the dark-field signal is modified by attenuation or phase shifts. We present a theoretical model for this group of effects and verify it by comparison to the measurements. These findings have significant implications for the interpretation of dark-field signal strength in polychromatic measurements.