The microbial deconstruction of the plant cell wall is a critical biological process, which also provides important substrates for environmentally sustainable industries. Enzymes that hydrolyze the plant cell wall generally contain non-catalytic carbohydrate binding modules (CBMs) that contribute to plant cell wall degradation. Here we report the biochemical properties and crystal structure of a family of CBMs (CBM60) that are located in xylanases. Uniquely, the proteins display broad ligand specificity, targeting xylans, galactans, and cellulose. Some of the CBM60s display enhanced affinity for their ligands through avidity effects mediated by protein dimerization. The crystal structure of vCBM60, displays a β-sandwich with the ligand binding site comprising a broad cleft formed by the loops connecting the two β-sheets. Ligand recognition at site 1 is, exclusively, through hydrophobic interactions, whereas binding at site 2 is conferred by polar interactions between a protein-bound calcium and the O2 and O3 of the sugar. The observation, that ligand recognition at site 2 requires only a β-linked sugar that contains equatorial hydroxyls at C2 and C3, explains the broad ligand specificity displayed by vCBM60. The ligand-binding apparatus of vCBM60 displays remarkable structural conservation with a family 36 CBM (CBM36); however, the residues that contribute to carbohydrate recognition are derived from different regions of the two proteins. Three-dimensional structure-based sequence alignments reveal that CBM36 and CBM60 are related by circular permutation. The biological and evolutionary significance of the mechanism of ligand recognition displayed by family 60 CBMs is discussed.