The graphene/Si heterojunction is attractive for high gain and broadband photodetection through photogating effect. However, the photoresponsivity in these devices are still limited to under 1 A W-1 if no narrowband absorption-enhanced nanostructures were used. In this paper, the effects of barriers on photoresponse are systematically studied at 1550 nm wavelength. Different barrier heights are obtained through selection of substrates, graphene doping and electrical tuning. Lower barrier height for graphene side and higher barrier height for silicon side are found to be beneficial for better infrared photoresponse. Through Polyetherimide doping of graphene and back-gated electrical modulation, the responsivity finally reached 5.71 A W-1, which to our knowledge is among the best results for graphene-based infrared photodetectors with graphene adopted as a light-absorption material. It is found that the thermionic emission efficiency of indirect transition in graphene is related to the difference in emissioin barrier height, and the lifetime of photoinduced carriers in the channel can be enhanced by built-in potential. These results lay the foundation for the photodetection applicatioins of graphene/Si heterojunction in the longer-wavelength infrared region.