The out-of-time-ordered correlators (OTOC), a fundamental concept for quantifying quantum information scrambling, has recently been suggested to be an order parameter to dynamically detect both equilibrium quantum phase transitions (EQPTs) and dynamical quantum phase transitions (DQPTs). Here we report the first experimental observation of EQPTs and DQPTs in a quantum spin chain via quench dynamics of OTOC on a nuclear magnetic resonance quantum simulator. We observe that the quench dynamics of the OTOC can unambiguously detect the DQPTs and the equilibrium critical point, while conventional order parameters such as the longitudinal magnetization can not. Moreover, we investigate the two-body correlations throughout the quench dynamics, and find that OTOC can extract the equilibrium critical point with higher accuracy and is more robust to decoherence than that of two-body correlation. Our experiment paves a way for experimentally investigating DQPTs through OTOCs and for studying the EQPTs through the nonequilibrium quantum quench dynamics with quantum simulators.