This study presents the application of the smoothed profile (SP) method to perform direct numerical simulations for the motion of both passive and active "squirming" particles in Newtonian and viscoelastic fluids. We found that fluid elasticity has a significant impact on both the transient behavior and the steady-state velocity of the particles. Specifically, we observe that the swirling flow generated by the squirmer's surface velocity significantly enhances their swimming speed as the Weissenberg number increases, regardless of the swimming type. Furthermore, we find that pushers outperform pullers in Oldroyd-B fluids, suggesting that the speed of a squirmer depends on the swimmer type. To understand the physical origin of the phenomenon of swirling flow enhancing the swimming speed, we investigate the velocity field and polymer conformation around non-swirling and swirling neutral squirmers in viscoelastic fluids. Our investigation reveals that the velocity field around the neutral swirling squirmers exhibits pusher-like extensional flow characteristics, as well as an asymmetric polymer conformation distribution, which gives rise to this increased propulsion. This is confirmed by the investigation of the force on a fixed squirmer, which revealed that the polymer stress, particularly its diagonal components, plays a critical role in enhancing the swimming speed of swirling squirmers in viscoelastic fluids. Additionally, our results demonstrate that the maximum swimming speeds of swirling squirmers occur at an intermediate value of the fluid viscosity ratio for all swimmer types. These findings have important implications for understanding the behavior of particles and micro-organisms in complex fluids.