Arrhythmia is a serious cardiovascular disease, and early diagnosis of arrhythmia is critical. In this study, we present a waveform-based signal processing (WBSP) method to produce state-of-the-art performance in arrhythmia classification. When performing WBSP, we first filtered ECG signals, searched local minima, and removed baseline wandering. Subsequently, we fit the processed ECG signals with Gaussians and extracted the parameters. Afterwards, we exploited the products of WBSP to accomplish arrhythmia classification with our proposed machine learning-based and deep learning-based classifiers. We utilized MIT-BIH Arrhythmia Database to validate WBSP. Our best classifier achieved 98.8% accuracy. Moreover, it reached 96.3% sensitivity in class V and 98.6% sensitivity in class Q, which both share one of the best among the related works. In addition, our machine learning-based classifier accomplished identifying four waveform components essential for automated arrhythmia classification: the similarity of QRS complex to a Gaussian curve, the sharpness of the QRS complex, the duration of and the area enclosed by P-wave.Clinical relevance- Early diagnosis and automated classification of arrhythmia is clinically essential.