A total of 138 animals from 7 species (Apodemus agrarius, Bandicota indica, Crocidura suaveolens, Mus caroli, Mus formosanus, Rattus losea, and Suncus murinus) captured in Taichung, located in central Taiwan, and Kinmen Island, an island off the shore of China, were examined for the presence of Anaplasma phagocytophilum. The presence of the bacteria, which causes human granulocytic anaplasmosis, was examined by nested PCR targeting the16S rDNA. Twelve animals (8.7%) from M. caroli and R. losea, and 25 (18.1%) from A. agrarius, B. indica, M. caroli, and R. losea were infected with A. phagocytophilum and Anaplasma bovis, respectively. Phylogenetic analysis revealed that partial 16S rDNA sequences in the 12 aforementioned animals showed higher similarity to the sequences related to A. phagocytophilum detected in wild rodents (Rattus and Niviventer) from southeast China. The sequences of the other 25 animals belonged to the A. bovis clade. We demonstrated that small wild mammals were infected with A. phagocytophilum and A. bovis in Taichung and Kinmen Island, Taiwan.