The fungal genus Colletotrichum contains hemibiotrophic phytopathogens being highly variable in host and tissue specificities. We sequenced a C. fructicola genome (1104-7) derived from an isolate of apple in China and compared it with the reference genome (Nara_gc5) derived from an isolate of strawberry in Japan. Mauve alignment and BlastN search identified 0.62 Mb lineage-specific (LS) genomic regions in 1104-7 with a length criterion of 10 kb. Genes located within LS regions evolved more dynamically, and a strongly elevated proportion of genes were closely related to non-Colletotrichum sequences. Two LS regions, containing nine genes in total, showed features of fungus-to-fungus horizontal transfer supported by both gene order collinearity and gene phylogeny patterns. We further compared the gene content variations among 13 Colletotrichum and 11 non-Colletotrichum genomes by gene function annotation, OrthoMCL grouping and CAFE analysis. The results provided a global evolutionary picture of Colletotrichum gene families, and identified a number of strong duplication/loss events at key phylogenetic nodes, such as the contraction of the detoxification-related RTA1 family in the monocot-specializing graminicola complex and the expansions of several ammonia production-related families in the fruit-infecting gloeosporioides complex. We have also identified the acquirement of a RbsD/FucU fucose transporter from bacterium by the Colletotrichum ancestor. In sum, this study summarized the pathogenic evolutionary features of Colletotrichum fungi at multiple taxonomic levels and highlights the concept that the pathogenic successes of Colletotrichum fungi require shared as well as lineage-specific virulence factors.