By performing molecular dynamics simulations with up to 132 million coarse-grained particles in half-micron sized boxes, we show that hydrodynamics quantitatively explains the finite-size effects on diffusion of lipids, proteins, and carbon nanotubes in membranes. The resulting Oseen correction allows us to extract infinite-system diffusion coefficients and membrane surface viscosities from membrane simulations despite the logarithmic divergence of apparent diffusivities with increasing box width. The hydrodynamic theory of diffusion applies also to membranes with asymmetric leaflets and embedded proteins, and to a complex plasma-membrane mimetic.