We consider imaging systems that measure the three first elements of the Stokes vector and deduce from them the degree of linear polarization and the angle of polarization. They require the acquisition of at least three intensity measurements, but performing more measurements is often thought to improve the estimation precision. We show that if the total acquisition time is fixed, the optimal number of measurements depends on the type of noise that affects the image: the estimation variance increases with the number of measurements N when the noise is additive; it is independent of N in the presence of Poisson shot noise and decreases with N when the angles of the analyzers fluctuate. In general, the optimal number of measurements results from a compromise on the robustness of these different types of perturbations.