The principles of algebraic image reconstruction are applied to THz computed tomography (THz-CT) in order to account for refraction within the sample. Using the nominal sample geometry as a priori knowledge, a highly accurate and robust image reconstruction algorithm based on the physics of geometric optics is presented. The validity of the geometric forward model is verified by a numerical simulation of Maxwell's equations. Furthermore, the developed method is experimentally tested using measurements performed with a fast THz-CT system based on a THz time-domain spectrometer in transmission mode. Automated evaluations of the reconstructed sample cross sections showed an accuracy of <150 μm.