Non-forest ecosystems (predominant in semi-arid and arid regions) contribute significantly to the increasing trend and interannual variation of land carbon uptake over the last three decades, yet the mechanisms are poorly understood. By analysing the flux measurements from 23 ecosystems in Australia, we found the the correlation between gross primary production (GPP) and ecosystem respiration (Re) was significant for non-forest ecosystems, but was not for forests. In non-forest ecosystems, both GPP and Re increased with rainfall, and, consequently net ecosystem production (NEP) increased with rainfall. In forest ecosystems, GPP and Re were insensitive to rainfall. Furthermore sensitivity of GPP to rainfall was dominated by the rainfall-driven variation of LAI rather GPP per unit LAI in non-forest ecosystems, which was not correctly reproduced by current land models, indicating that the mechanisms underlying the response of LAI to rainfall should be targeted for future model development.