We consider the Kuramoto-Sakaguchi model of identical coupled phase oscillators with a common noisy forcing. While common noise always tends to synchronize the oscillators, a strong repulsive coupling prevents the fully synchronous state and leads to a nontrivial distribution of oscillator phases. In previous numerical simulations, the formation of stable multicluster states has been observed in this regime. However, we argue here that because identical phase oscillators in the Kuramoto-Sakaguchi model form a partially integrable system according to the Watanabe-Strogatz theory, the formation of clusters is impossible. Integrating with various time steps reveals that clustering is a numerical artifact, explained by the existence of higher order Fourier terms in the errors of the employed numerical integration schemes. By monitoring the induced change in certain integrals of motion, we quantify these errors. We support these observations by showing, on the basis of the analysis of the corresponding Fokker-Planck equation, that two-cluster states are non-attractive. On the other hand, in ensembles of general limit cycle oscillators, such as Van der Pol oscillators, due to an anharmonic phase response function as well as additional amplitude dynamics, multiclusters can occur naturally.