This paper develops an empirical agent-based model to assess the impacts of Brexit on Scottish cattle farms. We first identify several trends and processes among Scottish cattle farms that were ongoing before Brexit: the lack of succession, the rise of leisure farming, the trend to diversify and industrialise, and, finally, the phenomenon of the "disappearing middle", characterised by the decline of medium-sized farms and the polarization of farm sizes. We then study the potential impact of Brexit amid the local context and those ongoing social processes. We find that the impact of Brexit is indeed subject to pre-Brexit conditions. For example, whether industrialization is present locally can significantly alter the impact of Brexit. The impact of Brexit also varies by location: we find a clear divide between constituencies in the north (highland and islands), the middle (the central belt) and the south. Finally, we argue that policy analysis of Brexit should consider the heterogeneous social context and the complex social processes under which Brexit occurs. Rather than fitting the world into simple system models and ignoring the evidence when it does not fit, we need to develop policy analysis frameworks that can incorporate real world complexities, so that we can assess the impacts of major events and policy changes in a more meaningful way.