Video-based monitoring of patients in the neonatal intensive care unit (NICU) has great potential for improving patient care. Video-based detection of clinical events, such as bottle feeding, would represent a step towards semi-automated charting of clinical events. Recording such events contemporaneously would address the limitations associated with retrospective charting. Such a system could also support oral feeding assessment tools, as the patient's feeding skills and nutrition are pivotal in monitoring their growth. We therefore leverage transfer learning using a pretrained VGG-16 model to classify images obtained during an intervention, to determine if a bottle-feeding event is occurring. Additionally, we explore a data expansion technique by extracting similar-feature images from publicly available sources to supplement our dataset of real NICU patients to address data scarcity. This work also visualizes and quantifies the gap between the source domain (ImageNet data subset) and target domain (NICU dataset), thereby illustrating the impact of expanding our training set for knowledge transfer proficiency. Results show an increase of over 18% in sensitivity after data expansion. Analysis of network activation maps indicates that data expansion is able to reduce the distance between the source and target domains.