The increasing number and quality of sensors integrated in mobile phones have paved the way for sensing schemes driven by city dwellers. The sensing quality can drastically depend on the mobile phone, and appropriate calibration strategies are needed. This paper evaluates the quality of noise measurements acquired by a variety of Android phones. The Ambiciti application was developed so as to acquire a larger control over the acquisition process. Pink and narrowband noises were used to evaluate the phones' accuracy at levels ranging from background noise to 90 dB(A) inside the lab. Conclusions of this evaluation lead to the proposition of a calibration strategy that has been embedded in Ambiciti and applied to more than 50 devices during public events. A performance analysis addressed the range, accuracy, precision, and reproducibility of measurements. After identification and removal of a bias, the measurement error standard deviation is below 1.2 dB(A) within a wide range of noise levels [45 to 75 dB(A)], for 12 out of 15 phones calibrated in the lab. In the perspective of citizens-driven noise sensing, in situ experiments were carried out, while additional tests helped to produce recommendations regarding the sensing context (grip, orientation, moving speed, mitigation, frictions, wind).