Widespread soil contamination with heavy metals has fostered the need for plant breeders to develop new crops that do not accumulate heavy metals. Metal-transporting transmembrane proteins that transport heavy metals across the plant plasma membrane are key targets for developing these new crops. Oryza sativa heavy metal ATPase 3 (OsHMA3) is known to be a useful gene for limiting cadmium (Cd) accumulation in rice. OsHMA2 is a close homolog of OsHMA3, but the function of OsHMA2 is unknown. To gain insight into the function of OsHMA2, we analyzed three Tos17 insertion mutants. The translocation ratios of zinc (Zn) and Cd were clearly lower in all mutants than in the wild type, suggesting that OsHMA2 is a major transporter of Zn and Cd from roots to shoots. By comparing each allele in the OsHMA2 protein structure and measuring the Cd translocation ratio, we identified the C-terminal region as essential for Cd translocation into shoots. Two alleles were identified as good material for breeding rice that does not contain Cd in the grain but does contain some Zn, and that grows normally.