The success of deep learning methods has enabled many modern wearable health applications, but has also highlighted the critical caveat of their extremely data hungry nature. While the widely explored wrist and finger photoplethysmography (PPG) sites are less affected, given the large available databases, this issue is prohibitive to exploring the full potential of novel recording locations such as in-ear wearables. To this end, we assess the feasibility of transfer learning from finger PPG to in-ear PPG in the context of deep learning for respiratory monitoring. This is achieved by introducing an encoder-decoder framework which is set up to extract respiratory waveforms from PPG, whereby simultaneously recorded gold standard respiratory waveforms (capnography, impedance pneumography and air flow) are used as a training reference. Next, the data augmentation and training pipeline is examined for both training on finger PPG and the subsequent fine tuning on in-ear PPG. The results indicate that, through training on two large finger PPG data sets (95 subjects) and then retraining on our own small in-ear PPG data set (6 subjects), the model achieves lower and more consistent test error for the prediction of the respiratory waveforms, compared to training on the small in-ear data set alone. This conclusively demonstrates the feasibility of transfer learning from finger PPG to in-ear PPG, leading to better generalisation across a wide range of respiratory rates.