The coherence of an individual quantum state can be meaningfully discussed only when referring to a preferred basis. This arbitrariness can, however, be lifted when considering sets of quantum states. Here we introduce the concept of set coherence for characterizing the coherence of a set of quantum systems in a basis-independent way. We construct measures for quantifying set coherence of sets of quantum states as well as quantum measurements. These measures feature an operational meaning in terms of discrimination games and capture precisely the advantage offered by a given set over incoherent ones. Along the way, we also connect the notion of set coherence to various resource-theoretic approaches recently developed for quantum systems.