Quantitative ultrasound (QUS) techniques based on the backscatter coefficient (BSC) aim to characterize the scattering properties of biological tissues. A scattering model is fit to the measured BSC and the fitted QUS parameters can provide local tissue microstructure, namely scatterer size and acoustic concentration. However, these techniques may fail to provide a correct description of tissue microstructure when the medium is polydisperse and/or dense. The objective of this study is to investigate the effects of scatterer size polydispersity in sparse or dense media on the QUS estimates. Four scattering models (i.e. the monodisperse and polydisperse sparse models, and the monodisperse and polydisperse concentrated models based on the structure factor) are compared to assess their accuracy and reliability in quantifying the QUS estimates. Simulations are conducted with different scatterer size distributions for sparse, moderately dense and dense media (volume fractions of 1%, 20% and 73%, respectively). The QUS parameters are estimated by using model-based inverse methods at different center frequencies between 8 MHz and 50 MHz. Experimental data are also analyzed using colon adenocarcinoma HT29 cell pellet biophantoms to further validate the results obtained from simulations at the volume fraction of 73%. Our findings reveal that the choice of scattering model has a significant impact on the accuracy of QUS estimates. For sufficiently high frequencies and dense media, the polydisperse concentrated model outperforms the other models and enables more accurate quantification. Furthermore, our results contribute to advancing our understanding of the complexities associated with scatterer size polydispersity and dense media in spectral-based QUS techniques.