Self-propelled swimmers such as bacteria agglomerate into clusters as a result of their persistent motion. In 1D, those clusters do not coalesce macroscopically and the stationary cluster size distribution (CSD) takes an exponential form. We develop a minimal lattice model for active particles in narrow channels to study how clustering is affected by the interplay between self-propulsion speed diversity and confinement. A mixture of run-and-tumble particles with a distribution of self-propulsion speeds is simulated in 1D. Particles can swap positions at rates proportional to their relative self-propulsion speed. Without swapping, we find that the average cluster size Lc decreases with diversity and follows a non-arithmetic power mean of the single-component Lc's, unlike the case of tumbling-rate diversity previously studied. Effectively, the mixture is thus equivalent to a system of identical particles whose self-propulsion speed is the harmonic mean self-propulsion speed of the mixture. With swapping, particles escape more quickly from clusters. As a consequence, Lc decreases with swapping rates and depends less strongly on diversity. We derive a dynamical equilibrium theory for the CSDs of binary and fully polydisperse systems. Similarly to the clustering behaviour of one-component models, our qualitative results for mixtures are expected to be universal across active matter. Using literature experimental values for the self-propulsion speed diversity of unicellular swimmers known as choanoflagellates, which naturally differentiate into slower and faster cells, we predict that the error in estimating their Lcvia one-component models which use the conventional arithmetic mean self-propulsion speed is around 30%.