This study investigated the development of fungal community composition in arable soil during the degradation of straw residue. We explored the short-term responses of the fungal community over 28 days of decomposition in soil using culture-independent polymerase chain reaction in combination with a clone library and denaturing gradient gel electrophoresis (DGGE). Fungal cellobiohydrolase I (cbhI) genes in the soil were also characterized, and their diversity suggested the existence of a different cellulose decomposer. The DGGE profiles based on fungal internal transcribed spacer analysis showed different successions of fungal populations during residue decomposition. Members of Lecythophora and Sordariales were dominant in the early succession, while Hypocrea and Engyodontium were better adapted in the late succession. The succession of fungal communities might be related to changes of residue quality during decomposition. Collectively, sequences assigned to Ascomycota members were dominant at different stages of the fungal succession during decomposition, revealing that they were key drivers responsible for residue degradation in the arable soil tested.