We present here the measurement of the diffusivity of spherical particles closely confined by narrow microchannels. Our experiments yield a two-dimensional map of the position-dependent diffusion coefficients parallel and perpendicular to the channel axis with a resolution down to 129 nm. The diffusivity was measured simultaneously in the channel interior, the bulk reservoirs, as well as the channel entrance region. In the channel interior we found strongly anisotropic diffusion. While the perpendicular diffusion coefficient close to the confining walls decreased down to approximately 25% of the value on the channel axis, the parallel diffusion coefficient remained constant throughout the entire channel width. In addition to the experiment, we performed finite element simulations for the diffusivity in the channel interior and found good agreement with the measurements. Our results reveal the distinctive influence of strong confinement on Brownian motion, which is of significance to microfluidics as well as quantitative models of facilitated membrane transport.