The attraction between electrons and holes in semiconductors forms excitons, which largely determine the optical properties of the hosting material, and hence the device performance, especially for low-dimensional systems. Mono- and few-layer black phosphorus (BP) are emerging two-dimensional (2D) semiconductors. Despite its fundamental importance and technological interest, experimental investigation of exciton physics has been rather limited. We report the first systematic measurement of exciton binding energies in ultrahigh-quality few-layer BP by infrared absorption spectroscopy, with layer (L) thickness ranging from 2 to 6 layers. Our experiments allow us to determine the exciton binding energy, decreasing from 213 meV (2L) to 106 meV (6L). The scaling behavior with layer numbers can be well described by an analytical model, which takes into account the nonlocal screening effect. Extrapolation to free-standing monolayer yields a large binding energy of ~800 meV. Our study provides insights into 2D excitons and their crossover from 2D to 3D, and demonstrates that few-layer BP is a promising high-quality optoelectronic material for potential infrared applications.