Carbohydrate binding modules (CBMs) are specialized proteins that bind to polysaccharides and oligosaccharides. Caldanaerobius polysaccharolyticus Man5ACBM16-1/CBM16-2 bind to glucose-, mannose-, and glucose/mannose-configured substrates. The crystal structures of the two proteins represent the only examples in CBM family 16, and studies that evaluate the roles of amino acid residues in ligand binding in this family are lacking. In this study, we probed the roles of amino acids (selected based on CBM16-1/ligand co-crystal structures) on substrate binding. Two tryptophan (Trp-20 and Trp-125) and two glutamine (Gln-81 and Gln-93) residues are shown to be critical in ligand binding. Additionally, several polar residues that flank the critical residues also contribute to ligand binding. The CBM16-1 Q121E mutation increased affinity for all substrates tested, whereas the Q21G and N97R mutants exhibited decreased substrate affinity. We solved CBM/substrate co-crystal structures to elucidate the molecular basis of the increased substrate binding by CBM16-1 Q121E. The Gln-121, Gln-21, and Asn-97 residues can be manipulated to fine-tune ligand binding by the Man5A CBMs. Surprisingly, none of the eight residues investigated was absolutely conserved in CBM family 16. Thus, the critical residues in the Man5A CBMs are either not essential for substrate binding in the other members of this family or the two CBMs are evolutionarily distinct from the members available in the current protein database. Man5A is dependent on its CBMs for robust activity, and insights from this study should serve to enhance our understanding of the interdependence of its catalytic and substrate binding modules.