A terahertz time-domain spectrometer is employed to study different birefringent samples. We develop a method based on the temporal waveform and the impulse response of a sample to map the anisotropy of their inner structure. To validate our algorithm, we study the polarization-affecting structure of various classes of materials such as crystals, plastics, and natural products. Among all samples we observe the largest birefringence for a rutile crystal with Deltan=3.3 at 1 THz.