Adipose tissue produces factors, including adipokines, cytokines and chemokines which, when released, systemically exert endocrine effects on multiple tissues thereby affecting their physiology. Adipokines also affect the hypothalamic-pituitary-gonadal (HPG) axis both centrally, at the hypothalamic-pituitary level, and peripherally acting on the gonads themselves. Among the adipokines, leptin, adiponectin, resistin, chemerin and the peptide kisspeptin have pleiotropic actions on the HPG axis affecting male and female fertility. Furthermore, adipokines and adipose tissue-produced factors readily affect the immune system resulting in inflammation, which in turn impact the HPG axis, thus evidencing a link between metabolic inflammation and fertility. In this review we provide an overview of the existing extensive bibliography on the crosstalk between adipose tissue-derived factors and the HPG axis, with particular focus on the impact of obesity and the metabolic syndrome on gonadal function and fertility.