A new method for imaging ultrafast dynamics in condensed matter using inelastic X-ray scattering (IXS) is described. Using the concepts of causality and irreversibility a general solution to the inverse scattering problem (or "phase problem") for IXS is illustrated, which enables direct imaging of dynamics of the electron density with resolutions of approximately 1 attosecond (10(-18) s) in time and <1 A in space. This method is not just Fourier transformation of the IXS data, but a means to impose causality on the data and reconstruct the charge propagator. The method can also be applied to inelastic electron or neutron scattering. A general outline of phenomena that can and cannot be studied with this technique and an outlook for the future is provided.