Atomic and molecular samples reduced to temperatures below one microkelvin, yet still in the gas phase, afford unprecedented energy resolution in probing and manipulating the interactions between their constituent particles. As a result of this resolution, atoms can be made to scatter resonantly on demand, through the precise control of a magnetic field. For simple atoms, such as alkalis, scattering resonances are extremely well characterized. However, ultracold physics is now poised to enter a new regime, where much more complex species can be cooled and studied, including magnetic lanthanide atoms and even molecules. For molecules, it has been speculated that a dense set of resonances in ultracold collision cross-sections will probably exhibit essentially random fluctuations, much as the observed energy spectra of nuclear scattering do. According to the Bohigas-Giannoni-Schmit conjecture, such fluctuations would imply chaotic dynamics of the underlying classical motion driving the collision. This would necessitate new ways of looking at the fundamental interactions in ultracold atomic and molecular systems, as well as perhaps new chaos-driven states of ultracold matter. Here we describe the experimental demonstration that random spectra are indeed found at ultralow temperatures. In the experiment, an ultracold gas of erbium atoms is shown to exhibit many Fano-Feshbach resonances, of the order of three per gauss for bosons. Analysis of their statistics verifies that their distribution of nearest-neighbour spacings is what one would expect from random matrix theory. The density and statistics of these resonances are explained by fully quantum mechanical scattering calculations that locate their origin in the anisotropy of the atoms' potential energy surface. Our results therefore reveal chaotic behaviour in the native interaction between ultracold atoms.