We study time evolution of velocity and pressure gradients in isotropic turbulence by quantifying their autocorrelation functions and decorrelation time scales. The Lagrangian analysis uses data in a public database generated by direct numerical simulation at a Reynolds number Re{lambda} approximately 433. It is confirmed that when averaging over the entire domain, correlation functions decay on time scales on the order of the global Kolmogorov turnover time scale. However, when performing the analysis in different subregions of the flow, turbulence intermittency leads to large spatial variability in the decay time scales. Remarkably, excellent collapse of the autocorrelation functions is recovered when using a locally defined Kolmogorov time scale. This provides new evidence for the validity of Kolmogorov's refined similarity hypothesis, but from a Lagrangian viewpoint that provides a natural frame to describe the dynamics of turbulence.