Mounting evidence shows that organophosphate flame retardants (OPFRs), especially aryl- and halogenated-OPFRs, exert various adverse health effects on living organisms. This study evaluated the hepatotoxic effect of trihexyl phosphate (THP) as a long-chain alkyl-OPFR on human hepatocyte cells (LO2) and mouse hepatocyte cells (AML12) by performing screening of cytotoxicity in vitro. In combination with transcriptomic analysis, toxicological mechanisms in vitro were further investigated. Results showed that THP triggered hepatotoxicity in vitro by altering four signaling pathways: endoplasmic reticulum (ER) stress, apoptosis, cell cycle, and the glycolysis signaling pathway. Exposure of LO2 and AML12 liver cells to THP (25 μg/mL) significantly induced ER stress-mediated apoptosis and cell cycle arrest. Meanwhile, downregulation of glycolysis caused the blockage of energy metabolism. Furthermore, the high performance liquid chromatography-quadrupole time-of-flight mass spectrometry (HPLC-Q-TOF-MS/MS) revealed that much of THP was absorbed into the cells and displayed stability in the two liver cell lines. In vivo assays using a mouse model demonstrated that exposure to THP at 400 mg/kg induced the ballooning degeneration of hepatocytes in liver tissue, whereas exposure to THP at 800 mg/kg caused acute liver injury with high alanine aminotransferase levels. This study provides novel insights into the impact of THP on hepatotoxicity in vitro and in vivo and uncovers the underlying toxicological mechanisms, which may serve as a guide for further ecological risk assessment and reasonable application of alkyl-OPFRs.