Semen infertility or sub-fertility, whether in humans or livestock species, remains a major concern for clinicians and technicians involved in reproduction. Indeed, they can cause tragedies in human relationships or have a dramatic overall negative impact on the sustainability of livestock breeding. Understanding and predicting semen fertility issues is therefore crucial and quality control procedures as well as biomarkers have been proposed to ensure sperm fertility. However, their predictive values appeared to be too limited and additional relevant biomarkers are still required to diagnose sub-fertility efficiently. During the last decade, the study of molecular mechanisms involved in spermatogenesis and sperm maturation highlighted the regulatory role of a variety of small non-coding RNAs (sncRNAs) and led to the discovery that sperm sncRNAs comprise both remnants from spermatogenesis and post-testicular sncRNAs acquired through interactions with extracellular vesicles along epididymis. This has led to the hypothesis that sncRNAs may be a source of relevant biomarkers, associated either with sperm functionality or embryo development. This review aims at providing a synthetic overview of the current state of knowledge regarding implication of sncRNA in spermatogenesis defects and their putative roles in sperm maturation and embryo development, as well as exploring their use as fertility biomarkers.