The striking in-plane anisotropy remains one of the most intriguing properties for the newly rediscovered black phosphorus (BP) 2D crystals. However, because of its rather low-energy band gap, the optical anisotropy of few-layer BP has been primarily investigated in the near-infrared (NIR) regime. Moreover, the essential physics that determine the intrinsic anisotropic optical property of few-layer BP, which is of great importance for practical applications in optical and optoelectronic devices, are still in the fancy of theory. Herein, we report the direct observation of the optical anisotropy of few-layer BP in the visible regime simply by using polarized optical microscopy. On the basis of the Fresnel equation, the intrinsic anisotropic complex refractive indices (n-iκ) in the visible regime (480-650 nm) were experimentally obtained for the first time using the anisotropic optical contrast spectra. Our findings not only provide a convenient approach to measure the optical constants of 2D layered materials but also suggest a possibility to design novel BP-based photonic devices such as atom-thick light modulators, including linear polarizer, phase plate, and optical compensator in a broad spectral range extending to the visible window.