The interest in development of methods and tools for recognizing human emotions has increased continuously. Using physiological information, especially the peripheral physiological signals, to identify emotions is an important direction for this area. This paper proposes an approach for emotion recognition based on energy-related features extracted from peripheral physiological signals. Three emotions: calm, happiness and fear, were elicited in 54 volunteers using video clips while three peripheral physiological signals were recorded: Electrocardiography (ECG), Photoplethysmography (PPG) and Respiration. Given that energy-related features of physiological signals are closely related to autonomic nervous systems activities, nine energy-related features were extracted from the recorded physiological signals. To find the optimal feature subset to represent the target emotions, the correlation between features and emotion state, as well as the discrimination ability of feature for emotion recognition were both analyzed. Four optimal features were then selected for further classification. Moreover, models based on Decision Tree (DT) were built to evaluate the performance of these features for purpose of recognition of emotion states of calm, happiness, and fear. The results show that the DT models based on these four optimal features could distinguish fear from calm (AUC=0.879, Accuracy=87.8%), happiness from calm (AUC=0.915, Accuracy=91.8%), and fear from happiness (AUC=0.822, Accuracy=81.8%), with a global recognition accuracy of 70.8%. These results indicate that energy-related features of peripheral physiological signals can reliably identify emotions, especially intense emotions.