This paper focuses on the use of an attention-based encoder-decoder model for the task of breathing sound segmentation and detection. This study aims to accurately segment the inspiration and expiration of patients with pulmonary diseases using the proposed model. Spectrograms of the lung sound signals and labels for every time segment were used to train the model. The model would first encode the spectrogram and then detect inspiratory or expiratory sounds using the encoded image on an attention-based decoder. Physicians would be able to make a more precise diagnosis based on the more interpretable outputs with the assistance of the attention mechanism.The respiratory sounds used for training and testing were recorded from 22 participants using digital stethoscopes or anti-noising microphone sets. Experimental results showed a high 92.006% accuracy when applied 0.5 second time segments and ResNet101 as encoder. Consistent performance of the proposed method can be observed from ten-fold cross-validation experiments.