Articular cartilage is a remarkable material able to sustain millions of loading cycles over decades of use outperforming any synthetic substitute. Crucially, how extracellular matrix constituents alter mechanical performance, particularly in shear, remains poorly understood. Here, we present experiments and theory in support of a rigidity percolation framework that quantitatively describes the structural origins of cartilage's shear properties and how they arise from the mechanical interdependence of the collagen and aggrecan networks making up its extracellular matrix. This framework explains that near the cartilage surface, where the collagen network is sparse and close to the rigidity threshold, slight changes in either collagen or aggrecan concentrations, common in early stages of cartilage disease, create a marked weakening in modulus that can lead to tissue collapse. More broadly, this framework provides a map for understanding how changes in composition throughout the tissue alter its shear properties and ultimate in vivo function.