The Olsen phosphorus (P) concentration of a soil is a key index that can be used to evaluate the P supply capacity of the soil and to estimate the optimal P fertilization rate. A study of the relationship between the soil Olsen P concentration and the P balance (P input minus P output) and their variations among different fertilization patterns will help to provide useful information for proper management of P fertilization. In this paper, the two investigated long-term experiments were established on black soils in the northeast region of China. Six fertilization treatments were selected: (1) unfertilized (CK); (2) nitrogen only (N); (3) nitrogen and potassium (NK); (4) nitrogen and phosphorus (NP); (5) nitrogen, phosphorus, and potassium (NPK); and (6) nitrogen, phosphorus, potassium and manure (NPKM). The results showed that the average Olsen P concentrations in the black soils at Gongzhuling and Harbin (16- and 31-year study periods, respectively), decreased by 0.49 and 0.56 mg kg-1 a-1, respectively, without P addition and increased by 3.17 and 1.78 mg kg-1 a-1, respectively, with P fertilization. The changes in soil Olsen P concentrations were significantly (P<0.05) correlated with the P balances at both sites except for the NP and NPK treatments at Gongzhuling. Under an average deficit of 100 kg ha-1 P, the soil Olsen P concentration at both sites decreased by 1.36~3.35 mg kg-1 in the treatments without P addition and increased by 4.80~16.04 mg kg-1 in the treatments with 100 kg ha-1 of P accumulation. In addition, the changes in Olsen P concentrations in the soil with 100 kg ha-1of P balance were significantly correlated with the P activation coefficient (PAC, percentage of Olsen P to total P, r=0.99, P<0.01) and soil organic matter content (r=0.91, P<0.01). A low pH was related to large changes of Olsen P by 1 kg ha-1 of P balance. These results indicated that soil organic matter and pH have important effects on the change in soil Olsen P by 1 kg ha-1 of P balance.