There are two common challenges in particle swarm optimization (PSO) research, that is, selecting proper exemplars and designing an efficient learning model for a particle. In this article, we propose a triple archives PSO (TAPSO), in which particles in three archives are used to deal with the above two challenges. First, particles who have better fitness (i.e., elites) are recorded in one archive while other particles who offer faster progress, called profiteers in this article, are saved in another archive. Second, when breeding each dimension of a potential exemplar for a particle, we choose a pair of elite and profiteer from corresponding archives as two parents to generate the dimension value by ordinary genetic operators. Third, each particle carries out a specific learning model according to the fitness of its potential exemplars. Furthermore, there is no acceleration coefficient in TAPSO aiming to simplify the learning models. Finally, if an exemplar has excellent performance, it will be regarded as an outstanding exemplar and saved in the third archive, which can be reused by inferior particles aiming to enhance the exploitation and to save computing resources. The experimental results and comparisons between TAPSO and other eight PSOs on 30 benchmark functions and four real applications suggest that TAPSO attains very promising performance in different types of functions, contributing to both higher solution accuracy and faster convergence speed. Furthermore, the effectiveness and efficiency of these new proposed strategies are discussed based on extensive experiments.