We study the nonequilibrium statistical mechanics of a system of freely moving particles, in which binary encounters lead either to an elastic collision or to the disappearance of the pair. Such a system of ballistic annihilation therefore constantly loses particles. The dynamics of perturbations around the free decay regime is investigated using the spectral properties of the linearized Boltzmann operator, which characterize linear excitations on all time scales. The linearized Boltzmann equation is solved in the hydrodynamic limit by a projection technique, which yields the evolution equations for the relevant coarse-grained fields and expressions for the transport coefficients. We finally present the results of molecular dynamics simulations that validate the theoretical predictions.