A handheld approach to 3D photoacoustic imaging is essential in clinical applications. To this end, we develop a 3D handheld photoacoustic imager for dynamic (temporally and spatially) volumetric visualization. In this 3D imager, the optically transmitting part and the acoustically receiving part are integrated into a single handheld probe with a compact size about 160 mm ×64 mm ×40 mm. Besides, a dedicated imaging reconstruction algorithm for the heterogeneous medium is developed based on the phase-shift migration method in the frequency domain, which deals well with the stratified condition in the designed system. Dynamic 3D imaging supporting flexible handheld operation is demonstrated with needle biopsy and in vitro temperature measurement for photothermal therapy. The development of such a 3D handheld photoacoustic system paves the way for compact and handheld-operating implementations, and its further clinical exploration is promising.