Altruistic punishment following social norm violations promotes human cooperation. However, experimental evidence indicates that some forms of punishment are spiteful rather than altruistic. Using two types of punishment games and seven non-strategic games, we identified strong behavioural differences between altruistic and spiteful punishers. Altruistic punishers who rejected unfair offers in the ultimatum game and punished norm violators in the third-party punishment game behaved pro-socially in various non-strategic games. Spiteful punishers who rejected unfair offers in the ultimatum game but did not punish norm violators in the third-party punishment game behaved selfishly in non-strategic games. In addition, the left caudate nucleus was larger in spiteful punishers than in altruistic punishers. These findings are in contrast to the previous assumption that altruistic punishers derive pleasure from enforcement of fairness norms, and suggest that spiteful punishers derive pleasure from seeing the target experience negative consequences.