Humans and the environment form a single complex system where humans not only influence ecosystems but also react to them. Despite this, there are far fewer coupled human-environment system (CHES) mathematical models than models of uncoupled ecosystems. We argue that these coupled models are essential to understand the impacts of social interventions and their potential to avoid catastrophic environmental events and support sustainable trajectories on multi-decadal timescales. A brief history of CHES modelling is presented, followed by a review spanning recent CHES models of systems including forests and land use, coral reefs and fishing and climate change mitigation. The ability of CHES modelling to capture dynamic two-way feedback confers advantages, such as the ability to represent ecosystem dynamics more realistically at longer timescales, and allowing insights that cannot be generated using ecological models. We discuss examples of such key insights from recent research. However, this strength brings with it challenges of model complexity and tractability, and the need for appropriate data to parameterize and validate CHES models. Finally, we suggest opportunities for CHES models to improve human-environment sustainability in future research spanning topics such as natural disturbances, social structure, social media data, model discovery and early warning signals. This article is part of the theme issue 'Ecological complexity and the biosphere: the next 30 years'.
Keywords: coupled human–environment systems; regime shifts; social learning; social norms; socio-ecological systems.