We present the exact adiabatic theory for the dynamics of the inhomogeneous density distribution of a classical fluid. Erroneous particle number fluctuations of dynamical density functional theory are absent, both for canonical and grand canonical initial conditions. We obtain the canonical free energy functional, which yields the adiabatic interparticle forces of overdamped Brownian motion. Using an exact and one of the most advanced approximate hard core free energy functionals, we obtain excellent agreement with simulations. The theory applies to finite systems in and out of equilibrium.