Fetal exposure to caffeine is associated with adverse pregnancy outcomes. Animal and human studies suggest that caffeine may have effects on the developing reproductive system. Here we report on mothers' smoking, coffee and alcohol use, recorded during pregnancy, and semen quality in sons in the age group of 38-47 years. Subjects were a subset of the Child Health and Development Studies, a pregnancy cohort enrolled between 1959 and 1967 in the Kaiser Foundation Health Plan near Oakland, California. In 2005, adult sons participated in a follow-up study (n = 338) and semen samples were donated by 196 participants. Samples were analyzed for sperm concentration, motility and morphology according to the National Cooperative Reproductive Medicine Network (Fertile Male Study) Protocol. Mean sperm concentration was reduced by approximately 16 million sperms for sons with high prenatal exposure (5 cups of maternal coffee use per day) compared with unexposed sons (P-value for decreasing trend = 0.09), which translates to a proportionate reduction of 25%. Mean percent motile sperm decreased by approximately 7 points (P-value = 0.04), a proportionate decline of 13%, and mean percent sperm with normal morphology decreased by approximately 2 points (P-value = 0.01), a proportionate decline of 25%. Maternal cigarette and alcohol use were not associated with son's semen quality. Adjusting for son's contemporary coffee, alcohol and cigarette use did not explain the maternal associations. Findings for son's coffee intake and father's prenatal coffee, cigarette and alcohol use were non-significant and inconclusive. These results contribute to the evidence that maternal coffee use during pregnancy may impair the reproductive development of the male fetus.