The aim of this study is to explore the pattern of change in multiple measures of cognitive abilities in a sample of oldest-old adults, comparing two different time metrics (chronological age and time to death) and therefore examining both underlying conceptual assumptions (age-related change and terminal decline). Moreover, the association with individual characteristics as sex, education, and dementia diagnosis was also examined. Measures of cognitive status (Mini-Mental State Examination and the Swedish Clock Test) and tests of crystallized (knowledge and synonyms), memory (verbal memory, nonverbal long-term memory, recognition and correspondence, and short-term memory), and visuospatial ability were included. The sample consisted of 671 older Swedish adult participants of the OCTO Twin Study. Linear mixed models with random coefficients were used to analyse change patterns and BIC indexes were used to compare models. Results showed that the time to death model was the best option in analyses of change in all the cognitive measures considered (except for the Information Test). A significant cognitive decline over time was found for all variables. Individuals diagnosed with dementia had lower scores at the study entrance and a faster decline. More educated individuals performed better in all the measures of cognition at study entry than those with poorer education, but no differences were found in the rate of change. Differences were found in age, sex, or time to death at baseline across the different measures. These results support the terminal decline hypothesis when compared to models assuming that cognitive changes are driven by normative aging processes.