Ultrafast electron diffraction (UED) enables studies of structural dynamics at atomic length and timescales, i.e., 0.1 nm and 0.1 ps, in single-shot mode. At present UED experiments are based on femtosecond laser photoemission from solid state cathodes. These photoemission sources perform excellently, but are not sufficiently bright for single-shot studies of, for example, biomolecular samples. We propose a new type of electron source, based on near-threshold photoionization of a laser-cooled and trapped atomic gas. The electron temperature of these sources can be as low as 10 K, implying an increase in brightness by orders of magnitude. We investigate a setup consisting of an ultracold electron source and standard radio-frequency acceleration techniques by GPT tracking simulations. The simulations use realistic fields and include all pairwise Coulomb interactions. We show that in this setup 120 keV, 0.1 pC electron bunches can be produced with a longitudinal emittance sufficiently small for enabling sub-100 fs bunch lengths at 1% relative energy spread. A transverse root-mean-square normalized emittance of epsilon(x) = 10 nm is obtained, significantly better than from photoemission sources. Correlations in transverse phase-space indicate that the transverse emittance can be improved even further, enabling single-shot studies of biomolecular samples.