Electrocardiography (ECG) is a standard diagnostic tool for evaluating the overall heart's electrical activity and is vital for detecting many cardiovascular diseases. Classifying ECG recordings using deep neural networks has been investigated in literature and has shown very good performance. However, this performance assumes that the training data is centralized, which is often not the case in real-life scenarios, where data resides in multiple places and only a small portion of it is labeled. Therefore, in this work, we propose an ECG classification system that focuses on preserving data privacy and enhancing overall system efficiency. We analyzed the complexity of previously proposed deep learning-based models and showed that the temporal convolutional network-based models (TCN) were the most efficient. Then, we built on the TCN models a modified split-learning (SL) system that achieves the same classification performance as the basic SL but reduces the communication overhead between the server and the client by 71.7% as well as reducing the computations at the client by 46.5% compared to the original SL system based on the TCN network. Finally, we implement semi-supervised learning in our system to enhance its classification performance by 9.1%-15.7%, when the training data consists only of 10% labeled data. We have tested our proposed system on a test IoT setup and it achieved satisfactory classification accuracy while being private and energy efficient for green-AI applications.