Generation and acceleration of energetic positrons based on laser plasma have attracted intense attention due to their potential applications in medical physics, high energy physics, astrophysics and nuclear physics. However, such compact positron sources face a series of challenges including the beam dispersion, dephasing and unstability. Here, we propose a scheme that couples the all-optical generation of electron-positron pairs and rapid acceleration of copious positrons in the terahertz (THz) field. In the scheme, nanocoulomb-scale electrons are first captured in the wakefield and accelerated to 2.5 GeV. Then these energetic electrons emit strong THz radiation when they go through an aluminum foil. Subsequently, abundant γ photons and positrons are generated during the collision of GeV electron beam and the scattering laser. Due to the strong longitudinal acceleration field and the transvers confining field of the emitted THz wave, the positrons can be efficiently accelerated to 800 MeV, with the peak beam brilliance of 2.26 × 1012s-1mm-2mrad-2eV-1. This can arouse potential research interests from PW-class laser facilities together with a GeV electron beamline.