Dyadic interactions often involve a dynamic process of mutual reciprocity; to steer a series of exchanges towards a desired outcome, both interactants must adapt their own behaviour according to that of their interaction partner. Understanding the brain processes behind such bidirectional reciprocity is therefore central to social neuroscience, but this requires measurement of both individuals' brains during real-world exchanges. We achieved this by performing functional magnetic resonance imaging (fMRI) on pairs of male individuals simultaneously while they interacted in a modified iterated Ultimatum Game (iUG). In this modification, both players could express their intent and maximise their own monetary gain by reciprocating their partner's behaviour - they could promote generosity through cooperation and/or discourage unfair play with retaliation. By developing a novel model of reciprocity adapted from behavioural economics, we then show that each player's choices can be predicted accurately by estimating expected utility (EU) not only in terms of immediate payoff, but also as a reaction to their opponent's prior behaviour. Finally, for the first time we reveal that brain signals implicated in social decision making are modulated by these estimates of EU, and become correlated more strongly between interacting players who reciprocate one another.