In this paper, we focus on the development of new methods suitable for efficient and reliable coarse-graining of non-equilibrium molecular systems. In this context, we propose error estimation and controlled-fidelity model reduction methods based on Path-Space Information Theory, combined with statistical parametric estimation of rates for non-equilibrium stationary processes. The approach we propose extends the applicability of existing information-based methods for deriving parametrized coarse-grained models to Non-Equilibrium systems with Stationary States. In the context of coarse-graining it allows for constructing optimal parametrized Markovian coarse-grained dynamics within a parametric family, by minimizing information loss (due to coarse-graining) on the path space. Furthermore, we propose an asymptotically equivalent method-related to maximum likelihood estimators for stochastic processes-where the coarse-graining is obtained by optimizing the information content in path space of the coarse variables, with respect to the projected computational data from a fine-scale simulation. Finally, the associated path-space Fisher Information Matrix can provide confidence intervals for the corresponding parameter estimators. We demonstrate the proposed coarse-graining method in (a) non-equilibrium systems with diffusing interacting particles, driven by out-of-equilibrium boundary conditions, as well as (b) multi-scale diffusions and the corresponding stochastic averaging limits, comparing them to our proposed methodologies.