This paper proposes accurate respiratory rate estimation using nasal breath sound recordings from a smartphone. Specifically, the proposed method detects nasal airflow using a built-in smartphone microphone or a headset microphone placed underneath the nose. In addition, we also examined if tracheal breath sounds recorded by the built-in microphone of a smartphone placed on the paralaryngeal space can also be used to estimate different respiratory rates ranging from as low as 6 breaths/min to as high as 90 breaths/min. The true breathing rates were measured using inductance plethysmography bands placed around the chest and the abdomen of the subject. Inspiration and expiration were detected by averaging the power of nasal breath sounds. We investigated the suitability of using the smartphone-acquired breath sounds for respiratory rate estimation using two different spectral analyses of the sound envelope signals: The Welch periodogram and the autoregressive spectrum. To evaluate the performance of the proposed methods, data were collected from ten healthy subjects. For the breathing range studied (6-90 breaths/min), experimental results showed that our approach achieves an excellent performance accuracy for the nasal sound as the median errors were less than 1% for all breathing ranges. The tracheal sound, however, resulted in poor estimates of the respiratory rates using either spectral method. For both nasal and tracheal sounds, significant estimation outliers resulted for high breathing rates when subjects had nasal congestion, which often resulted in the doubling of the respiratory rates. Finally, we show that respiratory rates from the nasal sound can be accurately estimated even if a smartphone's microphone is as far as 30 cm away from the nose.