Living organisms often display adaptive strategies that allow them to move efficiently even in strong confinement. With one single degree of freedom, the angle of a rotating bundle of flagella, bacteria provide one of the simplest examples of locomotion in the living world. Here we show that a purely physical mechanism, depending on a hydrodynamic stability condition, is responsible for a confinement induced transition between two swimming states in E. coli. While in large channels bacteria always crash onto confining walls, when the cross section falls below a threshold, they leave the walls to move swiftly on a stable swimming trajectory along the channel axis. We investigate this phenomenon for individual cells that are guided through a sequence of micro-fabricated tunnels of decreasing cross section. Our results challenge current theoretical predictions and suggest effective design principles for microrobots by showing that motility based on helical propellers provides a robust swimming strategy for exploring narrow spaces.