There is evidence to suggest that testosterone loss constitutes a risk for cognitive decline and possibly dementia, and that elderly men might benefit from exogenous supplementation of testosterone. Studies in non-human animals repeatedly report neuroexcitatory and neuroprotective properties of testosterone and enhanced memory performance after acute or chronic treatment. Positive effects of testosterone supplementation in older men have been reported in several, but not all, studies and require replication in larger randomized clinical trials before recommendations for clinical practice can be made. The current review summarizes recent studies on the neurobiological connection between testosterone and cognitive function in humans and non-human animals. When appropriate, we use the hippocampus as a model structure given it's involvement in sexually dymorphic spatial ability and sensitivity to both androgens and aging. In addition, a number of potential explanations of the discrepancy between data obtained in humans and non-human animals are discussed.