Alfalfa is a good green manure source, but its effect on rice growth has not been fully elucidated. Two green manure species, alfalfa and broad bean (Vicia faba L.), and two N fertilizer levels, alone or combination, were applied to a rice field. The results indicated that alfalfa had more pronounced effects on increasing soil labile phosphorus (P) fractions (including NaHCO3-Pi, NaOH-Pi), P uptake and soil enzyme activities (dehydrogenase, urease, acid phosphatase and β-glucosidase) than broad bean and N fertilizer. The transformation of NaHCO3-Po to labile P regulated by alfalfa played a significant direct and indirect effect on grain yield. Although a much lower N input from alfalfa addition, a similar grain yield with N fertilizer treatment was achieved, and the integration of alfalfa with N fertilizer produced the highest grain yield and P availability, which was associated with the highest urease, acid phosphatase and β-glucosidase activity in soil. These results indicate that alfalfa green manure had a great ability of increasing grain yield through enhancing P availability in rice paddy, which could give us a way to reduce N fertilizer application by enhancing P availability.