Free-breathing cardiac magnetic resonance (CMR) imaging has short examination time with high reproducibility. Detection of the end-diastole and the end-systole frames of the free-breathing cardiac magnetic resonance, supplemented by visual identification, is time consuming and laborious. We propose a novel method for automatic identification of both the end-diastole and the end-systole frames, in the free-breathing CMR imaging. The proposed technique utilizes the convolutional neural network to locate the left ventricle and to obtain the end-diastole and the end-systole frames from the respiratory motion signal. The proposed procedure works successfully on our free-breathing CMR data, and the results demonstrate a high degree of accuracy and stability. Convolutional neural network improves the postprocessing efficiency greatly and facilitates the clinical application of the free-breathing CMR imaging.