Glucose-stimulated insulin secretion (GSIS) is essential for blood glucose homeostasis and is impaired in type 2 diabetes mellitus. Understanding the regulatory components of GSIS has clinical implications for diabetes treatment. In this study, we found that olfactomedin 4 (OLFM4) is endogenously expressed in pancreatic islet β cells and further investigated its potential roles in glucose homeostasis and the pathogenesis of type 2 diabetes using mouse models. Olfm4-deficient mice showed significantly improved glucose tolerance and significantly increased insulin levels after glucose challenge compared with wild-type (WT) mice. GSIS, mitochondrial ATP production, and mitochondrial respiration were all significantly increased in islets isolated from Olfm4-deficient mice compared with those isolated from WT mice. In a high-fat diet (HFD)-induced diabetic mouse model, the increase in insulin levels after glucose challenge was significantly higher in Olfm4-deficient mice compared with WT mice. The impaired glucose tolerance and insulin resistance in HFD-fed mice were improved by loss of Olfm4. Olfm4 was found to be mainly localized in the mitochondria and interacts with GRIM-19 (a gene associated with retinoid-interferon mortality) in Min6 pancreatic β cells. Collectively, these studies suggest that Olfm4 negatively regulates GSIS. OLFM4 may represent a potential therapeutic target for impaired glucose tolerance and patients with type 2 diabetes.