Neonatal epileptic seizures take place in the early childhood years, accounting for a severe condition with several deaths and neurological problems in newborn neonates. Despite the early advancements on the diagnosis and/or treatment of this condition, as a major difficulty accounts the inability of the physicians to identify and characterize a seizure, as one a small percentage gets detected in neonatal intensive care units (NICU). An important step towards any kind of seizure classification is the detection and reduction of non-cerebral activity. Towards this direction, our multi-feature approach contains spectral and statistical characteristics of EEG signals of 79 infants with suspicion of seizure and assesses the performance of two classification algorithms iteratively. The trained models (Support Vector Machine (SVM) and Random Forest classifiers) yielded high classification performance (>80% and >85% respectively). A robust neonatal seizure classification scheme is thus proposed, along with nine high scoring spectrum and statistical features. The importance of embedding an artefact reduction approach is also discussed, since the complex artifacts spread throughout the signals have great impact on the accuracy of the algorithms. The nine extracted high scoring spectral and statistical features might be used as potential biomarkers for neonatal seizure prediction in a clinical setting.