The field of medical acoustics is gaining constantly-increasing attention by the scientific community, with the the general goal being the automatic understanding of medical related signals to assist medical personnel in the decision making process. In this direction, this work introduces a framework able to differentiate between normal and abnormal respiratory sounds by learning relationships characterizing pairs of sounds. More specifically, considering the nature of respiratory sounds, we designed a feature set able to capture the coarse and fine structure exhibited such signals by means of multiresolution analysis. Similar/dissimilar relationships are modeled via a suitably-learned Siamese Neural Network encompassing a series of convolutional layers. Interestingly, such a relationship learning framework conveniently solves the existing class imbalance problem as it is trained on pairs of similar/dissimilar audio signals. Importantly, we employed the dataset designed for the IEEE BioCAS 2022 Grand challenge on Respiratory Sound Classification along with a standardized experimental protocol allowing reproducibility and reliable comparison between different approaches. After extensive experiments assessing the proposed framework from diverse points of view, including an ablation study, it is shown that it outperforms existing approaches, while providing explainable predictions via a Q&A scheme allowing interaction with the medical experts.