Strategic interactions arise in all domains of life. This form of competition often plays out in dynamically changing environments. The strategies employed in a population may alter the state of the environment, which may in turn feedback to change the incentive structure of strategic interactions. Feedbacks between strategies and the environment are common in social-ecological systems, evolutionary-ecological systems, and even psychological-economic systems. Here we develop a framework of 'eco-evolutionary game theory' that enables the study of strategic and environmental dynamics with feedbacks. We consider environments governed either by intrinsic growth, decay, or tipping points. We show how the joint dynamics of strategies and the environment depend on the incentives for individuals to lead or follow behavioral changes, and on the relative speed of environmental versus strategic change. Our analysis unites dynamical phenomena that occur in settings as diverse as human decision-making, plant nutrient acquisition, and resource harvesting. We discuss implications in fields ranging from ecology to economics.