Ultrasonography is one of the main imaging methods for diagnosing thyroid nodules. Automatic differentiation between benign and malignant nodules in ultrasound images can greatly assist inexperienced clinicians in their diagnosis. The key of problem is the effective utilization of the features of ultrasound images. In this study, we propose a method that is based on the combination of conventional ultrasound and ultrasound elasticity images based on a convolutional neural network and introduces richer feature information for the classification of benign and malignant thyroid nodules. First, the conventional network model performs pretraining on ImageNet and transfers the feature parameters to the ultrasound image domain by transfer learning so that depth features may be extracted and small samples may be processed. Then, we combine the depth features of conventional ultrasound and ultrasound elasticity images to form a hybrid feature space. Finally, the classification is completed on the hybrid feature space, and an end-to-end CNN model is implemented. The experimental results demonstrate that the accuracy of the proposed method is 0.9470, which is better than that of other single data-source methods under the same conditions.