Recently, terahertz (THz)-driven particle accelerators have drawn increasing attention. The development of high-energy-gain THz accelerators on chip has been a challenge. Here we propose a concept of an on-chip THz-driven particle accelerator that uses few-cycle THz pulses to drive dielectric prisms. It avoids the serious waveguide dispersion of previous THz linacs based on dielectric lined waveguides and enhances the electron-energy gain. In addition, we propose to use prism stacks to overcome the asynchronization effect when accelerating low-energy particles, by which a longer acceleration length with even higher energy gain can be realized. Compared with the available on-chip dielectric laser accelerators, the proposed scheme avoids serious dielectric dispersion and enhances accelerated bunch charge. Hence, it promises an attractive particle accelerator on chip.