This paper presents an algorithm for ultrafast ultrasound localization microscopy (ULM) used for the detection, localization, accumulation, and rendering of intravenously injected ultrasound contrast agents (UCAs) enabling to yield hemodynamic maps of the brain microvasculature. It consists in integrating a robust principal component analysis (RPCA)-based approach into the ULM process for more robust tissue filtering, resulting in more accurate ULM images. Numerical experiments conducted on an in vivo rat brain perfusion dataset demonstrate the efficiency of the proposed approach compared to the most widely used state-of-the-art method.