We have studied nanoscale diffusion of membrane hydration water in fluid-phase lipid bilayers made of 1,2-dimyristoyl-3-phosphocholine (DMPC) using incoherent quasi-elastic neutron scattering. Dynamics were fit directly in the energy domain using the Fourier transform of a stretched exponential. By using large, 2-dimensional detectors, lateral motions of water molecules and motions perpendicular to the membranes could be studied simultaneously, resulting in 2-dimensional maps of relaxation time, τ, and stretching exponent, β. We present experimental evidence for anomalous (sub-diffusive) and anisotropic diffusion of membrane hydration water molecules over nanometer distances. By combining molecular dynamics and Brownian dynamics simulations, the potential microscopic origins for the anomaly and anisotropy of hydration water were investigated. Bulk water was found to show intrinsic sub-diffusive motion at time scales of several picoseconds, likely related to caging effects. In membrane hydration water, however, the anisotropy of confinement and local dynamical environments leads to an anisotropy of relaxation times and stretched exponents, indicative of anomalous dynamics.