How does a lack of sleep affect our brains? In contrast to the benefits of sleep, frameworks exploring the impact of sleep loss are relatively lacking. Importantly, the effects of sleep deprivation (SD) do not simply reflect the absence of sleep and the benefits attributed to it; rather, they reflect the consequences of several additional factors, including extended wakefulness. With a focus on neuroimaging studies, we review the consequences of SD on attention and working memory, positive and negative emotion, and hippocampal learning. We explore how this evidence informs our mechanistic understanding of the known changes in cognition and emotion associated with SD, and the insights it provides regarding clinical conditions associated with sleep disruption.