We proposed and demonstrated a coarse-fine method to achieve fast locating of external vibration for the phase-sensitive optical time-domain reflectometer (φ-OTDR) sensing system. Firstly, the acquired backscattered traces from heterodyne coherent φ-OTDR systems are spatially divided into a few segments along a sensing fiber for coarse locating, and most of the acquired data can be excluded by comparing the phase difference between the endpoints in adjacent segments. Secondly, the amplitude-based locating is implemented within the target segments for fine locating. By using the proposed coarse-fine locating method, we have numerically and experimentally investigated a distributed vibration sensor based on the heterodyne coherent φ-OTDR system with a 50-km-long sensing fiber. We find that the computation cost of signal processing for locating is significantly reduced in the long-haul sensing fiber, showing a potential application in real-time locating of external vibration.