We consider a noninteracting many-fermion system populating levels of a unitary random matrix ensemble (equivalent to the q=2 complex Sachdev-Ye-Kitaev model)-a generic model of single-particle quantum chaos. We study the corresponding many-particle level statistics by calculating the spectral form factor analytically using algebraic methods of random matrix theory, and match it with an exact numerical simulation. Despite the integrability of the theory, the many-body spectral rigidity is found to have a surprisingly rich landscape. In particular, we find a residual repulsion of distant many-body levels stemming from single-particle chaos, together with islands of level attraction. These results are encoded in an exponential ramp in the spectral form factor, which we show to be a universal feature of nonergodic many-fermion systems embedded in a chaotic medium.