The Olsen phosphorus (Olsen-P) concentration of soil is generally a good indicator for estimating the bioavailability of P and environmental risk in soils. To maintain soil Olsen-P at adequate levels for crop growth and environmental sustainability, the relationship between soil Olsen-P and the P budget (the P input minus the output) as well as the variations of soil Olsen-P and P budget were investigated from three long-term (22 years) experiments in China. Five treatments were selected: (1) unfertilized control (CK); (2) nitrogen and potassium (NK); (3) nitrogen, phosphorous, and potassium (NPK); (4) nitrogen, phosphorous, potassium and straw; (5) nitrogen, phosphorous, potassium and manure. The results showed that without P fertilizers (CK, NK), there was a soil P deficit of 75-640 kg ha-1, and the lowest P deficit (mean of CK and NK) was in Eutric Cambisol. Soil Olsen-P decreased by 0.11-0.39 mg kg-1 year-1 in the order of Luvic Phaeozems > Eutric Cambisol > Calcaric Cambisol. Soil Olsen-P and the P deficit had a significantly (P<0.01) positive linear relationship. For every 100 kg of P ha-1 of deficit, soil Olsen-P decreased by 0.44-9.19 mg kg-1 in the order of Eutric Cambisol > Luvic Phaeozems > Calcaric Cambisol. Under the P fertilizer treatments (NPK, NPKS, and NPKM), soil Olsen-P showed an obvious surplus (except the NPK and NPKS in Luvic Phaeozems) of 122-2190 kg ha-1, and the largest P surplus was found under the NPKM treatment at each site. The relation between soil Olsen-P and the experimental years could be simulated using quadratic equation of one unknown in Calcaric Cambisol for the lower P input after 14 years of fertilization. And soil Olsen-P increased by 1.30-7.69 mg kg-1 year-1 in the order of Luvic Phaeozems > Eutric Cambisol. The relation between soil Olsen-P and the P surplus could be simulated by a simple linear equation except under NPK and NPKS in Luvic Phaeozems. With 100 kg ha-1 P surplus, soil Olsen-P increased by 3.24-7.27 mg kg-1 in the order of Calcaric Cambisol (6.42 mg kg-1) > Eutric Cambisol (3.24 mg kg-1). In addition, the change in soil Olsen-P with a 100 kg P ha-1 surplus (soil Olsen-P efficiency) was affected by the soil organic matter (SOM), pH, and CaCO3 content, etc. In the practice of fertilization, it's not necessary to increase the amount of P fertilizers, farmers should take measure to solve the local problem, for adjust the soil pH of Eutric Cambisol and Calcaric Cambisol, and apply more nitrogen in Luvic Phaeozems. In the area of serious soil P surplus, it is encouraged to stop applying P fertilizers for a few years to take advantage of soil accumulated P and make the high Olsen-P content decrease to a reasonable level.