Allowing triplet components of individual geminals, spin-contaminated strongly orthogonal geminal wave functions may emerge, which can be ameliorated by spin-projection techniques. Of the latter, half-projection was previously shown to be useful, offering a compromise between the amount of remaining spin-contamination and the violation of size consistency generated by projection. This paper investigates how a half-projected spin-contaminated geminal wave function can be improved by multi-configuration perturbation theory to incorporate dynamical correlation effects.