Containing the spreading of crime is a major challenge for society. Yet, since thousands of years, no effective strategy has been found to overcome crime. To the contrary, empirical evidence shows that crime is recurrent, a fact that is not captured well by rational choice theories of crime. According to these, strong enough punishment should prevent crime from happening. To gain a better understanding of the relationship between crime and punishment, we consider that the latter requires prior discovery of illicit behavior and study a spatial version of the inspection game. Simulations reveal the spontaneous emergence of cyclic dominance between "criminals", "inspectors", and "ordinary people" as a consequence of spatial interactions. Such cycles dominate the evolutionary process, in particular when the temptation to commit crime or the cost of inspection are low or moderate. Yet, there are also critical parameter values beyond which cycles cease to exist and the population is dominated either by a stable mixture of criminals and inspectors or one of these two strategies alone. Both continuous and discontinuous phase transitions to different final states are possible, indicating that successful strategies to contain crime can be very much counter-intuitive and complex. Our results demonstrate that spatial interactions are crucial for the evolutionary outcome of the inspection game, and they also reveal why criminal behavior is likely to be recurrent rather than evolving towards an equilibrium with monotonous parameter dependencies.