Glycoside hydrolases that degrade plant cell walls have complex molecular architectures in which one or more catalytic modules are appended to noncatalytic carbohydrate-binding modules (CBMs). CBMs promote binding to polysaccharides and potentiate enzymic hydrolysis. Although there are diverse sequence-based families of xylan-binding CBMs, these modules, in general, recognize both decorated and unsubstituted forms of the target polysaccharide, and thus the evolutionary rationale for this diversity is unclear. Using immunohistochemistry to interrogate the specificity of six xylan-binding CBMs for their target polysaccharides in cell walls has revealed considerable differences in the recognition of plant materials between these protein modules. Family 2b and 15 CBMs bind to xylan in secondary cell walls in a range of dicotyledon species, whereas family 4, 6, and 22 CBMs display a more limited capability to bind to secondary cell walls. A family 35 CBM, which displays more restricted ligand specificity against purified xylans than the other five protein modules, reveals a highly distinctive binding pattern to plant material including the recognition of primary cell walls of certain dicotyledons, a feature shared with CBM15. Differences in the specificity of the CBMs toward walls of wheat grain and maize coleoptiles were also evident. The variation in CBM specificity for ligands located in plant cell walls provides a biological rationale for the repertoire of structurally distinct xylan-binding CBMs present in nature, and points to the utility of these modules in probing the molecular architecture of cell walls.