Ultrasound localization microscopy (ULM) overcomes the acoustic diffraction limit and enables the visualization of microvasculature at subwavelength resolution. However, challenges remain in ultrafast ULM implementation, where short data acquisition time, efficient data processing speed, and high imaging resolution need to be considered simultaneously. Recently, deep learning (DL)-based methods have exhibited potential in speeding up ULM imaging. Nevertheless, a certain number of ultrasound (US) data ( L frames) are still required to accumulate enough localized microbubble (MB) events, leading to an acquisition time within a time span of tens of seconds. To further speed up ULM imaging, in this article, we present a new DL-based method, termed as ULM-GAN. By using a modified conditional generative adversarial network (cGAN) framework, ULM-GAN is able to reconstruct a superresolution image directly from a temporal mean low-resolution (LR) image generated by averaging l -frame raw US images with l being significantly smaller than L . To evaluate the performance of ULM-GAN, a series of numerical simulations and phantom experiments are both implemented. The results of the numerical simulations demonstrate that when performing ULM imaging, ULM-GAN allows ∼ 40 -fold reduction in data acquisition time and ∼ 61 -fold reduction in computational time compared with the conventional Gaussian fitting method, without compromising spatial resolution according to the resolution scaled error (RSE). For the phantom experiments, ULM-GAN offers an implementation of ULM with ultrafast data acquisition time ( ∼ 0.33 s) and ultrafast data processing speed ( ∼ 0.60 s) that makes it promising to observe rapid biological activities in vivo.