Telomere length (TL) has long been associated with aging, as telomeres serve as protective caps of chromosomes, and are thus deeply involved in the preservation of genome integrity and are vital to cellular functions. Traditionally, a strong link connects aging and infertility in both sexes, with an earlier onset in females. Over the past decade, telomeres have attracted increasing attention due to the role they play in fertility. In this review, we investigated the potential positive or negative association between relative TL and different factors of female and male infertility. A systematic search of the PubMed database was conducted. Out of the 206 studies identified, 45 were reviewed as they fulfilled the criteria of validity and relevance. Following an analysis and a comparison of the study outcomes, several clear trends were observed. The majority of female infertility factors were associated with a shorter TL, with the exception of endometriosis, premature ovarian failure and clear cell carcinoma that were associated with a longer TL and polycystic ovary syndrome (PCOS), which revealed conflicting results among several studies, leading to ambiguous conclusions. Male infertility factors were associated with a shorter TL. Although this review can provide an outline of general trends in the association of TL with infertility factors, further epidemiological and original research studies are required to focus on investigating the basis of these varying lengths of telomeres.