Many pathological phenomena of male infertility are related to epigenetic changes in male germ cells. Epigenetic regulation during spermatogenesis plays an important role in mitotic/meiotic divisions and spermiogenesis. The histones have various post-translational modifications on different amino acid residues during spermatogenesis. These modifications are crucial to the precise regulation of spermatogenesis. Moreover, the histone-to-protamine transition will occur during spermiogenesis. Many studies have also found that abnormal changes of histone modifications during spermatogenesis may damage the sperm development, leading to male sterility. This article reviews the changes of histone modifications during spermatogenesis, the regulation of the development of male germ cells, and the relationship between histone abnormalities and male sterility.