In this work we investigate the generic properties of a stochastic linear model in the regime of high dimensionality. We consider in particular the vector autoregressive (VAR) model and the multivariate Hawkes process. We analyze both deterministic and random versions of these models, showing the existence of a stable phase and an unstable phase. We find that along the transition region separating the two regimes the correlations of the process decay slowly, and we characterize the conditions under which these slow correlations are expected to become power laws. We check our findings with numerical simulations showing remarkable agreement with our predictions. We finally argue that real systems with a strong degree of self-interaction are naturally characterized by this type of slow relaxation of the correlations.