Using naturalistic stimuli, we assessed the ability to infer what other people are feeling in three groups of participants: healthy elderly adults, patients suffering from the behavioral variant of frontotemporal dementia (FTD-b), and patients suffering from Alzheimer's disease (AD). After watching videotaped interviews of everyday people (nonactors) discussing an emotionally relevant event in their lives, participants answered questions regarding the interviewee's feelings. Both patient groups inferred emotions as accurately as the healthy elderly, provided the emotions were displayed unambiguously and consistently across the interview. However, when the displayed emotions became more variable and ambiguous, performance in both patient groups became impaired relative to healthy elderly participants. The similar profile across the two clinical groups despite their differences in social skills suggests that nonsocial cognitive processes affected in dementia may be an important factor in drawing inferences about other people's feelings.