A new formalism was recently proposed to improve random phase approximation (RPA) correlation energies by including approximate exchange effects [B. Mussard et al., J. Chem. Theory Comput. 12, 2191 (2016)]. Within this framework, by keeping only the electron-hole contributions to the exchange kernel, two approximations can be obtained: An adiabatic connection analog of the second order screened exchange (AC-SOSEX) and an approximate electron-hole time-dependent Hartree-Fock (eh-TDHF). Here we show how this formalism is suitable for an efficient implementation within the plane-wave basis set. The response functions involved in the AC-SOSEX and eh-TDHF equations can indeed be compactly represented by an auxiliary basis set obtained from the diagonalization of an approximate dielectric matrix. Additionally, the explicit calculation of unoccupied states can be avoided by using density functional perturbation theory techniques and the matrix elements of dynamical response functions can be efficiently computed by applying the Lanczos algorithm. As shown by several applications to reaction energies and weakly bound dimers, the inclusion of the electron-hole kernel significantly improves the accuracy of ground-state correlation energies with respect to RPA and semi-local functionals.