Optical coherence elastography (OCE) has been proposed for a range of clinical applications. However, the majority of these studies have been performed using bulky, lab-based imaging systems. A compact, handheld imaging probe would accelerate clinical translation, however, to date, this had been inhibited by the slow scan rates of compact devices and the motion artifact induced by the user's hand. In this paper, we present a proof-of-concept, handheld quantitative micro-elastography (QME) probe capable of scanning a 6 × 6 × 1 mm volume of tissue in 3.4 seconds. This handheld probe is enabled by a novel QME acquisition protocol that incorporates a custom bidirectional scan pattern driving a microelectromechanical system (MEMS) scanner, synchronized with the sample deformation induced by an annular PZT actuator. The custom scan pattern reduces the total acquisition time and the time difference between B-scans used to generate displacement maps, minimizing the impact of motion artifact. We test the feasibility of the handheld QME probe on a tissue-mimicking silicone phantom, demonstrating comparable image quality to a bench-mounted setup. In addition, we present the first handheld QME scans performed on human breast tissue specimens. For each specimen, quantitative micro-elastograms are co-registered with, and validated by, histology, demonstrating the ability to distinguish stiff cancerous tissue from surrounding soft benign tissue.