Where continental plates break apart, slip along multiple normal faults provides the required space for the Earth's crust to thin and subside. After initial rifting, however, the displacement on normal faults observed at the sea floor seems not to match the inferred extension. Here we show that crustal thinning can be accomplished in such extensional environments by a system of conjugate concave downward faults instead of multiple normal faults. Our model predicts that these concave faults accumulate large amounts of extension and form a very thin crust (< 10 km) by exhumation of mid-crustal and mantle material. This transitional crust is capped by sub-horizontal detachment surfaces over distances exceeding 100 km with little visible deformation. Our rift model is based on numerical experiments constrained by geological and geophysical observations from the Alpine Tethys and Iberia/Newfoundland margins. Furthermore, we suggest that the observed transition from broadly distributed and symmetric extension to localized and asymmetric rifting is directly controlled by the existence of a strong gabbroic lower crust. The presence of such lower crustal gabbros is well constrained for the Alpine Tethys system. Initial decoupling of upper crustal deformation from lower crustal and mantle deformation by progressive weakening of the middle crust is an essential requirement to reproduce the observed rift evolution. This is achieved in our models by the formation of weak ductile shear zones.