The accurate measurement of three-dimensional (3D) fiber orientation in the brain is crucial for reconstructing fiber pathways and studying their involvement in neurological diseases. Optical imaging methods such as polarization-sensitive optical coherence tomography (PS-OCT) provide important tools to directly quantify fiber orientation at micrometer resolution. However, brain imaging based on the optic axis by PS-OCT so far has been limited to two-dimensional in-plane orientation, preventing the comprehensive study of connectivity in 3D. In this work, we present a novel method to obtain the 3D fiber orientation in full angular space with only two illumination angles. We measure the optic axis orientation and the apparent birefringence by PS-OCT from a normal and a 15 deg tilted illumination, and then apply a computational method yielding the 3D optic axis orientation and true birefringence. We verify that our method accurately recovers a large range of through-plane orientations from -85 deg to 85 deg with a high angular precision. We further present 3D fiber orientation maps of entire coronal sections of human cerebrum and brainstem with 10 μm in-plane resolution, revealing unprecedented details of fiber configurations. We envision that further development of our method will open a promising avenue towards large-scale 3D fiber axis mapping in the human brain and other complex fibrous tissues at microscopic level.