A rapid increase in the number of patients with dementia has emerged as a global health challenge. Accumulating evidence suggests that early diagnosis and timely intervention can delay cognitive decline. The diagnosis of dementia is commonly performed using neuropsychological tests, such as the Mini-Mental State Examination (MMSE), administered by trained examiners. While these traditional neuropsychological tests are valid and reliable, they are neither simple nor sufficiently short as routine screening tools for dementia. Here, we developed a brief cognitive assessment utilizing an eye-tracking technology. The subject views a series of short (178 s) task movies and pictures displayed on a monitor while their gaze points are recorded by the eye-tracking device, and the cognitive scores are determined from the gaze plots data. The cognitive scores were measured by both an eye tracking-based assessment and neuropsychological tests in 80 participants, including 27 cognitively healthy controls (HC), 26 patients with mild cognitive impairment (MCI), and 27 patients with dementia. The eye tracking-based cognitive scores correlated well with the scores from the neuropsychological tests, and they showed a good diagnostic performance in detecting patients with MCI and dementia. Rapid cognitive assessment using eye-tracking technology can enable quantitative scoring and the sensitive detection of cognitive impairment.