This letter proposes an efficient scheme for the early diagnosis of bearing defects using a convolutional neural network (CNN) and energy distribution maps (EDMs) of acoustic emission spectra. The CNN automates the process of feature extraction from the EDM. The features learned by the CNN are used by an ensemble classifier, that is, a combination of a multilayer perceptron that is integral to typical CNN architectures and a support vector machine to diagnose bearing defects. The experimental results confirm that the proposed scheme diagnoses bearing defects more effectively than existing methods under variable speed conditions.