A key challenge in haptics is designing human-human communications involving touch to facilitate positive effects on social interactions. An important consideration in designing social touch is understanding the effect of social stimuli on perception, in addition to that of a physical stimulus, because social touch always involves a partner. This study presents an experiment to demonstrate that facial expressions induce haptic perception. We developed a human-agent interaction system on a display in which participants moved the mouse cursor to click the target icon while the agent behaved as if it pulled the cursor back in the opposite direction, showing either a negative or neutral face. The perceived force during the interaction was quantified by the control display ratio using a psychophysical approach. The results show that the negative face induced a significantly greater perceived force than the neutral face. In addition, the perceived force correlated with the individual's evaluation of the facial expression; that is, the more unpleasant or aroused they perceived the facial expression to be, the more force they perceived. This study sheds light on the design of social touch performed by people who have physical or mediated contact with each other in physical space or cyberspace.