Functional near infrared spectroscopy (fNIRS) is a neuroimaging technique that has grown vigorously in recent years. With noticeable attention, machine learning methods have also been applied to fNIRS. However, the current approach lacks interpretability of the results. In recent years, the utilization and investigation of fNIRS have experienced significant growth and are now being utilized in clinical research. However, the collection of clinical fNIRS data is limited in sample size. Therefore, our aim is to utilize the collected fNIRS data from all channels and achieve interpretable analysis results with minimal human manipulation, channel selection or feature extraction. We developed an fNIRS-based interpretable model and used class-specific gradient information to visualize the biomarkers captured by the model via locating the important region. The accuracy of our model's classification was 6% higher than that of the conventional SVM method under within-subject classification. The model focuses on signals from the left brain in the classification of right-hand finger tapping task, while in the task of classifying left-handed movements, the model relies on signals from the right brain. These results were consistent with current understanding of physiology.Clinical Relevance- The machine learning-based fNIRS model has the potential to be used for the diagnosis and prediction of therapeutic efficacy in clinical settings.