The fitness landscape of a locus, the array of fitnesses conferred by its alleles, can be affected by allele replacements at other loci, in the presence of epistatic interactions between loci. In a pair of diverging homologous proteins, the initially high probability that an amino acid replacement in one of them will make it more similar to the other declines with time, implying that the fitness landscapes of homologous sites diverge. Here, we use data on within-population non-synonymous polymorphisms and on amino acid replacements between species to study the dynamics, after an amino acid replacement, of the fitness of the ancestral amino acid, and show that selection against its restoration increases with time. This effect can be owing to increase of fitness conferred by the new amino acid occupying the site, and/or to decline of fitness conferred by the replaced amino acid. We show that the fitness conferred by the replaced amino acid rapidly declines, reaching a new lower steady-state level after approximately 20 per cent of amino acids in the protein get replaced. Therefore, amino acid replacements in evolving proteins are routinely involved in negative epistatic interactions with currently absent amino acids, and chisel off the unused parts of the fitness landscape.