It is challenging to increase the sensitivity of a hydrogen sensor operating at room temperature due to weak sorption and tiny mass of hydrogen. In this work, an ultrasonic sensor is presented for detecting hydrogen, which is composed of a 128° YX-LiNbO3 substrate and a reduced graphene oxide (RGO) sensitive layer with a platinum catalyzer. By optimizing the depositing parameters of RGO and platinum, a considerably high sensitivity is achieved at room temperature. A frequency shift of 308.9 kHz is obtained in 100 ppm hydrogen mixed with argon, and a frequency shift of 24.4 kHz is obtained in 1000 ppm hydrogen mixed in synthetic air. It is demonstrated that in addition to strong sorption of the sensitive layer, the coaction of mass load and conductivity variation is key to high sensitivity of the sensor. By establishing the original conductivity of the sensitive layer within the "conductivity window" for enhancing electrical response, we improve the sensitivity of the ultrasonic sensor, which is available for detecting hydrogen with an extremely low concentration of 5 ppm.