After entering the human body, drugs for treating diseases, which are prone to delivery and release in an uncontrolled manner, are affected by various factors. Based on this, many researchers utilize various microenvironmental changes encountered during drug delivery to trigger drug release and have proposed stimuli-responsive drug delivery systems. In recent years, metal-organic frameworks (MOFs) have become promising stimuli-responsive agents to release the loaded therapeutic agents at the target site to achieve more precise drug delivery due to their high drug loading, excellent biocompatibility, and high stimuli-responsiveness. The MOF-based stimuli-responsive systems can respond to various stimuli under pathological conditions at the site of the lesion, releasing the loaded therapeutic agent in a controlled manner, and improving the accuracy and safety of drug delivery. Due to the changes in different physical and chemical factors in the pathological process of diseases, the construction of stimuli-responsive systems based on MOFs has become a new direction in drug delivery and controlled release. Based on the background of the rapidly increasing attention to MOFs applied in drug delivery, we aim to review various MOF-based stimuli-responsive drug delivery systems and their response mechanisms to various stimuli. In addition, the current challenges and future perspectives of MOF-based stimuli-responsive drug delivery systems are also discussed in this review.