Two-dimensional (2D) Ruddlesden-Popper perovskites are currently drawing significant attention as highly-stable photoactive materials for optoelectronic applications. However, the insulating nature of organic ammonium layers in 2D perovskites results in poor charge transport and limited performance. Here, we demonstrate that Al2O3/2D perovskite heterostructure can be utilized as photoactive dielectric for high-performance MoS2 phototransistors. The type-II band alignment in 2D perovskites facilitates effective spatial separation of photo-generated carriers, thus achieving ultrahigh photoresponsivity of >108 A/W at 457 nm and >106 A/W at 1064 nm. Meanwhile, the hysteresis loops induced by ionic migration in perovskite and charge trapping in Al2O3 can neutralize with each other, leading to low-voltage phototransistors with negligible hysteresis and improved bias stress stability. More importantly, the recombination of photo-generated carriers in 2D perovskites depends on the external biasing field. With an appropriate gate bias, the devices exhibit wavelength-dependent constant photoresponsivity of 103-108 A/W regardless of incident light intensity.