Recently, it has become easier and more common to measure physiological signals through wearable devices such as smart watches. Extracting emotional states of individuals with problems expressing it, such as autistic individuals, can help their parents, friends, and therapists to obtain a better understanding of what they feel throughout their day. Although emotion recognition methods based on physiological signals have been studied for many years, there is a smaller body of literature about systems working with data obtained from wearable devices. In this paper, we present an emotion recognition system with a small footprint suitable for limited resources of wearable devices. Other than identifying the emotions (with a success rate of 65%), The proposed system also tags each recognition with a confidence value (on average 57%).