Ultrasonic measurement of distance is a well-known low cost method but only a few vibrometers have been developed because sensitivity, spatial resolution, and bandwidth are not high or wide enough for standard laboratory applications. Nevertheless, compared to optical vibrometers, two interesting properties should be considered: very low frequency noise (0.1 Hz to 1 kHz) is reduced and the long wavelength enables rough surfaces to be investigated. Moreover, the ultrasonic probe is a differential sensor, without being a mechanical load for the vibrating structure as usual accelerometers based on contacting transducers are. The main specificity of the presented probe is its ultralow noise electronics including a 3/2 order phase locked loop which extracts the phase modulation related to the amplitude of the detected vibration. This article presents the main useful physical aspects and details of the actual probe. The given application is the measurement of the vibration of an isolated optical bench excited at very low frequency with an electromagnetic transducer.