Fine root decomposition represents a large carbon (C) cost to plants, and serves as a potential soil C source, as well as a substantial proportion of net primary productivity. Coarse roots differ markedly from fine roots in morphology, nutrient concentrations, functions, and decomposition mechanisms. Still poorly understood is whether a consistent global pattern exists between the decomposition of fine (<2 mm root diameter) and coarse (≥2 mm) roots. A comprehensive terrestrial root decomposition dataset, including 530 observations from 71 sampling sites, was thus used to compare global patterns of decomposition of fine and coarse roots. Fine roots decomposed significantly faster than coarse roots in middle latitude areas, but their decomposition in low latitude regions was not significantly different from that of coarse roots. Coarse root decomposition showed more dependence on climate, especially mean annual temperature (MAT), than did fine roots. Initial litter lignin content was the most important predictor of fine root decomposition, while lignin to nitrogen ratios, MAT, and mean annual precipitation were the most important predictors of coarse root decomposition. Our study emphasizes the necessity of separating fine roots and coarse roots when predicting the response of belowground C release to future climate changes.