The endosome-associated deubiquitinase (DUB) AMSH is a member of the JAMM family of zinc-dependent metallo isopeptidases with high selectivity for Lys63-linked polyubiquitin chains, which play a key role in endosomal-lysosomal sorting of activated cell surface receptors. The catalytic domain of the enzyme features a flexible flap near the active site that opens and closes during its catalytic cycle. Structural analysis of its homologues, AMSH-LP (AMSH-like protein) and the fission yeast counterpart, Sst2, suggests that a conserved Phe residue in the flap may be critical for substrate binding and/or catalysis. To gain insight into the contribution of this flap in substrate recognition and catalysis, we generated mutants of Sst2 and characterized them using a combination of enzyme kinetics, X-ray crystallography, molecular dynamics simulations, and isothermal titration calorimetry (ITC). Our analysis shows that the Phe residue in the flap contributes key interactions during the rate-limiting step but not to substrate binding, since mutants of Phe403 exhibit a defect only in kcat but not in KM. Moreover, ITC studies show Phe403 mutants have similar KD for ubiquitin compared to the wild-type enzyme. The X-ray structures of both Phe403Ala and the Phe403Trp, in both the free and ubiquitin bound form, reveal no appreciable structural change that might impair substrate or alter product binding. We observed that the side chain of the Trp residue is oriented identically with respect to the isopeptide moiety of the substrate as the Phe residue in the wild-type enzyme, so the loss of activity seen in this mutant cannot be explained by the absence of a group with the ability to provide van der Waals interactions that facilitate the hyrdolysis of the Lys63-linked diubiquitin. Molecular dynamics simulations indicate that the flap in the Trp mutant is quite flexible, allowing almost free rotation of the indole side chain. Therefore, it is possible that these different dynamic properties of the flap in the Trp mutant, compared to the wild-type enzyme, manifest as a defect in interactions that facilitate the rate-limiting step. Consistent with this notion, the Trp mutant was able to cleave Lys48-linked and Lys11-linked diubiquitin better than the wild-type enzyme, indicating altered mobility and hence reduced selectivity.