The use of memory kernels stemming from a Mori-Zwanzig approach to time series analysis is discussed. We show that despite its success in determining properties from an analytical model, the kernel itself is not easily interpreted. We consider a recently introduced discretization of the kernel and show that its properties can be quite different from its continuous counterpart. We provide a rigorous analysis of the discrete case and show for several analytically calculated memory kernels of simple time series processes that their features are not readily detectable in the kernel. We show furthermore that practical relevant Mori-Zwanzig models with a finite kernel form a true subclass of the autoregressive moving average (ARMA) models. The fact that this approach already veils the properties of these simple time series gives rise to severe doubts about its applicability in more complex situations.