We study consequences of long-range elasticity in thermally assisted dynamics of yield stress materials. Within a two-dimensional mesoscopic model we calculate the mean-square displacement and the dynamical structure factor for tracer particle trajectories. The ballistic regime at short time scales is associated with a compressed exponential decay in the dynamical structure factor, followed by a subdiffusive crossover prior to the onset of diffusion. We relate this crossover to spatiotemporal correlations and thus go beyond established mean field predictions.