Cerenkov luminescence imaging (CLI) can provide information of medical radionuclides used in nuclear imaging based on Cerenkov radiation, which makes it possible for optical means to image clinical radionuclide labeled probes. However, the exceptionally weak Cerenkov luminescence (CL) from Cerenkov radiation is susceptible to lots of impulse noises introduced by high energy gamma rays generating from the decays of radionuclides. In this work, a temporal median filter is proposed to remove this kind of impulse noises. Unlike traditional CLI collecting a single CL image with long exposure time and smoothing it using median filter, the proposed method captures a temporal sequence of CL images with shorter exposure time and employs a temporal median filter to smooth a temporal sequence of pixels. Results of in vivo experiments demonstrated that the proposed temporal median method can effectively remove random pulse noises induced by gamma radiation and achieve a robust CLI image.