Most methods of optimal control cannot obtain accurate time-optimal protocols. The quantum brachistochrone equation is an exception, and has the potential to provide accurate time-optimal protocols for a wide range of quantum control problems. So far, this potential has not been realized, however, due to the inadequacy of conventional numerical methods to solve it. Here we show that the quantum brachistochrone problem can be recast as that of finding geodesic paths in the space of unitary operators. We expect this brachistochrone-geodesic connection to have broad applications, as it opens up minimal-time control to the tools of geometry. As one such application, we use it to obtain a fast numerical method to solve the brachistochrone problem, and apply this method to two examples demonstrating its power.