Major earthquakes frequently nucleate near the base of the seismogenic zone, close to the brittle-ductile transition. Fault zone rupture at greater depths is inhibited by ductile flow of rock. However, the microphysical mechanisms responsible for the transition from ductile flow to seismogenic brittle/frictional behaviour at shallower depths remain unclear. Here we show that the flow-to-friction transition in experimentally simulated calcite faults is characterized by a transition from dislocation and diffusion creep to dilatant deformation, involving incompletely accommodated grain boundary sliding. With increasing shear rate or decreasing temperature, dislocation and diffusion creep become too slow to accommodate the imposed shear strain rate, leading to intergranular cavitation, weakening, strain localization, and a switch from stable flow to runaway fault rupture. The observed shear instability, triggered by the onset of microscale cavitation, provides a key mechanism for bringing about the brittle-ductile transition and for nucleating earthquakes at the base of the seismogenic zone.