A noncontact photoacoustic imaging method based on optical quadrature detection is proposed. The photo-induced acoustic signal is detected by an optical method without contacting the specimen. By utilizing the intrinsic phase difference of a multiport optical interferometer, the quadrature signal of a conventional interferometric signal could be obtained. With this quadratic signal pair, we could reconstruct the photoacoustic signal without suffering from the initial phase drift that usually occurs in a conventional interferometric system. The performance of the proposed system is verified by imaging human hairs embedded in a polydimethylsiloxane resin block. The system's lateral and axial resolutions are measured to be 84 and 86 μm at a 1.5 mm depth of a PDMS resin block, respectively. The experimental result is good enough to distinguish the hairs staggered in depth.