Species distribution modelling (SDM) gained importance on biodiversity distribution and conservation studies worldwide, including prioritizing areas for public policies and international treaties. Useful for large-scale approaches and species distribution estimates, it is a plus considering that a minor fraction of the planet is adequately sampled. However, minimizing errors is challenging, but essential, considering the uses and consequences of such models. In situ validation of the SDM outputs should be a key-step-in some cases, urgent. Bioacoustics can be used to validate and refine those outputs, especially if the focal species' vocalizations are conspicuous and species-specific. This is the case of echolocating bats. Here, we used extensive acoustic monitoring (>120 validation points over an area of >758,000 km2, and producing >300,000 sound files) to validate MaxEnt outputs for six neotropical bat species in a poorly-sampled region of Brazil. Based on in situ validation, we evaluated four threshold-dependent theoretical evaluation metrics' ability in predicting models' performance. We also assessed the performance of three widely used thresholds to convert continuous SDMs into presence/absence maps. We demonstrated that MaxEnt produces very different outputs, requiring a careful choice on thresholds and modeling parameters. Although all theoretical evaluation metrics studied were positively correlated with accuracy, we empirically demonstrated that metrics based on specificity-sensitivity and sensitivity-precision are better for testing models, considering that most SDMs are based on unbalanced data. Without independent field validation, we found that using an arbitrary threshold for modelling can be a precarious approach with many possible outcomes, even after getting good evaluation scores. Bioacoustics proved to be important for validating SDMs for the six bat species analyzed, allowing a better refinement of SDMs in large and under-sampled regions, with relatively low sampling effort. Regardless of the species assessing method used, our research highlighted the vital necessity of in situ validation for SDMs.