Various pattern-recognition or machine learning-based methods have recently been developed to improve the accuracy of the motor imagery (MI)-based brain-computer interface (BCI). However, more research is needed to reduce the training time to apply it to the real-world environment. In this study, we propose a subject-transfer decoding method based on a convolutional neural network (CNN) which is robust even with a small number of training trials. The proposed CNN was pre-trained with other subjects' MI data and then fine-tuned to the target subject's training MI data. We evaluated the proposed method using the BCI competition IV data2a, which had the 4-class MIs. Consequently, on the same test dataset, with changing the number of training trials, the proposed method showed better accuracy than the self-training method, which used only the target subject's data for training, as averaged 86.54±7.78% (288 trials), 85.76 ±8.00% (240 trials), 84.65±8.11% (192 trials), and 83.29 ±8.25% (144 trials), respectively, which was 4.94% (288 trials), 6.10% (240 trials), 9.03% (192 trials), and 12.31% (144 trials)-point higher than the self-training method. Consequently, the proposed method was shown to be effective in maintaining classification accuracy even with the reduced training trials.