The binary defocusing fringe projection profilometry (FPP) technique has demonstrated various advantages for high-speed and high-accuracy three-dimensional (3D) surface measurement. However, higher fringe frequency does not necessarily give better measurements in binary defocusing FPP. To improve the 3D geometry measurement accuracy, this paper proposes an optimal frequency selection approach by analyzing the phase error distribution under different defocusing degrees. The phase error is analyzed theoretically based on the multi-frequency temporal phase unwrapping process, and the associated relationship with fringe frequency, system defocusing degree, noise, and other influencing factors is established. Meanwhile, optimal fringe frequency in a specific system is selected by the theoretical model combined with the validation of simulation experiments. Finally, the measurement accuracy could be effectively enhanced by the generated binary fringe patterns of optimal frequency. Both simulations and experiments verify the effectiveness and robustness of the proposed method.