Optical speckle patterns occur when a coherent optical wavefront is randomized but such stochastic yet deterministic information about the medium can be decoded. A simple setup is inspired to monitor the decorrelation of speckle patterns within the memory effect range when the medium is photoacoustically perturbated. Experimentally, a linear relationship is confirmed between the speckle correlation change and the peak-to-peak amplitude of the ultrasonic transducer-detected photoacoustic waves, and the detection sensitivity is comparable. Such a plain specklegram-based method may find special interests when no direct contact is allowed between the sample and the photoacoustic detector.