Emotions potentially have a significant impact on human actions and recognizing affective states is an effective way of implementing Brain-Computer Interface (BCI) systems which process brain signals to allow direct communication and interaction with the environment. In this paper, a real-time emotion recognition model was developed on the basis of physiological signals. A sensor fusion method is developed to detect human emotion by using data acquired from ElectroEncephaloGraphy (EEG) and ElectroDermal Activity (EDA) sensors. The proposed physiology-based emotion recognition system using a neural network was implemented and tested on human subjects, and a classification accuracy of 94% on three different emotions was achieved.