The affective role of touch has opened new perspectives in human-machine interaction. This paper presents an emotion recognition algorithm to investigate the role of tactile stimuli conveyed through a wearable haptic system during affective reading. To this end, a group of 32 healthy volunteers underwent an emotional stimulation by reading affective texts, with and without the concurrent presence of pleasant haptic stimuli. Throughout the experiment, autonomic nervous system dynamics was quantified through heart rate variability (HRV) and electrodermal activity (EDA) analyses. EDA and HRV features were then used as input of a SVM-RFE learning algorithm for an automatic recognition of neutral and arousing texts. The affective recognition of the reading was performed in the presence or absence of the haptic stimulation. Results show that the affective perception induced by the neutral and arousing reading were discriminated with a significantly improved accuracy (+14.5%) when a caress-like haptic stimulus was conveyed to the user.