Three dimensional information of the Gouy phase shift in a converging spherical terahertz (THz) beam is directly observed by using a THz balanced electro-optic holographic imaging system. The major properties of the Gouy phase shift are presented, including the longitudinal and transverse distributions, relationships with the frequency and the f-number, influence on the THz polarization. The imaging technique supplies an accurate and comprehensive measurement method for observing and understanding the Gouy phase shift.