Agricultural land uses can impact stream ecosystems by reducing suitable habitat, altering flows, and increasing inputs of diffuse pollutants including fine inorganic sediment (< 2 mm). These changes have been linked to altered community composition and declines in biodiversity. Determining the mechanisms driving stream biotic responses, particularly threshold impacts, has, however, proved elusive. To investigate a sediment threshold response by benthic invertebrates, an intensive survey of 30 agricultural streams was conducted along gradients of deposited sediment and dissolved nutrients. Partial redundancy analysis showed that invertebrate community composition changed significantly along the gradient of deposited fine sediment, whereas the effect of dissolved nitrate was weak. Pollution-sensitive invertebrates (%EPT, Ephemeroptera, Plecoptera, Trichoptera) demonstrated a strong nonlinear response to sediment, and change-point analysis indicated marked declines beyond a threshold of -20% fine sediment covering the streambed. Structural equation modeling indicated that decreased habitat availability (i.e., coarse substrate and associated interstices) was the key driver affecting pollution-sensitive invertebrates, with degraded riparian condition controlling resources through direct (e.g., inputs) and indirect (e.g., flow-mediated) effects on deposited sediment. The identification of specific effects thresholds and the underlying mechanisms (e.g., loss of habitat) driving these changes will assist managers in setting sediment criteria and standards to better guide stream monitoring and rehabilitation.