Optical anti-counterfeiting has been developed as a promising optical-sensing technique. A self-activated KGaSiO4 phosphor was successfully prepared using the traditional solid-state method. The photoluminescence spectra of the as-synthesized phosphors indicate that the ultra-narrow band emission with green light peak at 503 nm is obtained when phosphors are excited by 254 nm UV light. Additionally, the measured afterglow curve shows that the emission of this phosphor can last more than 1200 s after UV excitation stops, which indicates that KGaSiO4 is a potential candidate for anti-counterfeiting materials. The luminescent and decay mechanism are discussed by theoretical calculation and thermo-luminescent spectra in detail. The theoretical model can provide support for explaining the mechanism of narrow band or persistent phosphor.