High-k materials such as Al2O3 and HfO2 are widely used as gate dielectrics in graphene devices. However, the effective work function values of metal gate in graphene FET are significantly deviated from their vacuum work function, which is similar to the Fermi level pinning effect observed in silicon MOSFETs with high-k dielectric. The degree of deviation represented by a pinning factor was much worse with HfO2 (pinning factor (S) = 0.19) than with Al2O3 (S = 0.69). We propose that the significant pinning-like behaviors induced by HfO2 are correlated with the oxygen exchange reactions occurred at the interface of graphene and HfO2.