With the popularity of the wireless body sensor network, real-time and continuous collection of single-lead electrocardiogram (ECG) data becomes possible in a convenient way. Data mining from the collected single-lead ECG waves has therefore aroused extensive attention worldwide, where early detection of atrial fibrillation (AF) is a hot research topic. In this paper, a two-channel convolutional neural network combined with a data augmentation method is proposed to detect AF from single-lead short ECG recordings. It consists of three modules, the first module denoises the raw ECG signals and produces 9-s ECG signals and heart rate (HR) values. Then, the ECG signals and HR rate values are fed into the convolutional layers for feature extraction, followed by three fully connected layers to perform the classification. The data augmentation method is used to generate synthetic signals to enlarge the training set and increase the diversity of the single-lead ECG signals. Validation experiments and the comparison with state-of-the-art studies demonstrate the effectiveness and advantages of the proposed method.