Ultrasensitive detection of molecules by graphene plasmons based surface enhanced infrared absorption spectroscopy (SEIRAS) has attracted considerable research interest in recent years. However, SEIRAS still suffers from low enhancement. Herein, we investigated the crucial factors that determined the enhancement of graphene plasmons based SEIRAS. Through numerical calculations, it found that the enhancement of SEIRAS can be significantly improved by increasing the absorptance of graphene plasmons and the electron relaxation time of graphene. It revealed that such results were related to the mode energy of graphene plasmons. High absorptance and long electron relaxation time would result in high mode energy, which would in turn induce large local electric field to enhance the SEIRAS signal. Moreover, it showed that the resonant center of a molecular vibrational mode can be accurately extracted from the Rabi splitting spectra obtained by sweeping the Fermi energy of graphene. Our study could provide a guidance to improve the enhancement of graphene plasmons based SEIRAS for ultrasensitive molecular detection.