The field of dielectric laser accelerators (DLA) garnered a considerable interest in the past six years as it offers novel opportunities in accelerator science and potentially transformative applications. Currently, the most widespread approach considers silicon-based structures due to their low absorption and high refractive index in the infrared spectral region and the well-developed silicon processing technology. In this paper we investigate a diamond as an alternative to silicon, mainly due to its considerably higher damage threshold. In particular, we find that our diamond grating allows a three times higher acceleration gradient (60 MeV/m) compared to silicon gratings designed for a similar electron energy. Using more complex geometries, GeV/m acceleration gradients are within reach for subrelativistic electrons.