After a decade's efforts, a large amount of highly luminescent metal nanoparticles with different sizes and surface chemistries have been developed. While the luminescence is often attributed to particle size effects, other structural parameters such as surface ligands, valence states of metal atoms and crystallinity of nanoparticles also have a significant influence on emission properties and mechanisms. In this review, we summarized the strategies used to create luminescent gold nanoparticles with sizes from few to millions of atoms and discussed how these structural factors affect their photoluminescence.