Magnetic particle imaging (MPI) is a medical imaging modality of recent origin, and it exploits the nonlinear magnetization phenomenon to recover a spatially dependent concentration of nanoparticles. In practice, image reconstruction in MPI is frequently carried out by standard Tikhonov regularization with nonnegativity constraint, which is then minimized by a Kaczmarz type method. In this work, we revisit two issues in the numerical reconstruction in MPI in the lens of inverse theory, i.e. the choice of fidelity and acceleration, and propose two algorithmic tricks, i.e. a whitening procedure to incorporate the noise statistics and accelerating Kaczmarz iteration via randomized SVD. The two tricks are straightforward to implement and easy to incorporate in existing reconstruction algorithms. Their significant potentials are illustrated by extensive numerical experiments on a publicly available dataset.