Thanks to recent advances in digital stethoscopes and rapid adoption of deep learning techniques, there has been tremendous progress in the field of Computerized Auscultation Analysis (CAA). Despite these promising leaps, the deploy-ment of these technologies in real-world applications remains limited due to inherent challenges with properly interpreting clinical data, particularly auscultations. One of the limiting factors is the inherent ambiguity that comes with variability in clinical opinion, even from highly trained experts. The lack of unanimity in expert opinions is often ignored in developing machine learning techniques to automatically screen normal from abnormal lung signals, with most algorithms being developed and tested on highly curated datasets. To better understand the potential pitfalls this selective analysis could cause in deployment, the current work explores the impact of clinical opinion variability on algorithms to detect adventitious patterns in lung sounds when trained on gold-standard data. The study shows that uncertainty in clinical opinion introduces far more variability and performance drop than dissidence in expert judgments. The study also explores the feasibility of automatically flagging auscultation signals based on their estimated uncertainty, thereby recommending further reassessment as well as improving computer-aided analysis.