Interferon-inducible transmembrane (IFITM) protein family members IFITM1, -2, and -3 restrict the infection of multiple enveloped viruses. Significant enrichment of a minor IFITM3 allele was recently reported for patients who were hospitalized for seasonal and 2009 H1N1 pandemic flu. This IFITM3 allele lacks the region corresponding to the first amino-terminal 21 amino acids and is unable to inhibit influenza A virus. In this study, we found that deleting this 21-amino-acid region relocates IFITM3 from the endosomal compartments to the cell periphery. This finding likely underlies the lost inhibition of influenza A virus that completes its entry exclusively within endosomes at low pH. Yet, wild-type IFITM3 and the mutant with the 21-amino-acid deletion inhibit HIV-1 replication equally well. Given the pH-independent nature of HIV-1 entry, our results suggest that IFITM3 can inhibit viruses that enter cells via different routes and that its N-terminal region is specifically required for controlling pH-dependent viruses.