Accumulating evidence suggests that physical activity may be beneficial in preserving cognition in late life. This study examined the association between baseline and changes in physical activity and cognitive decline in community-dwelling older people. Data were from the Korean Longitudinal Study of Aging, with 2605 aged 65 years and older subjects interviewed in 2006 and followed up for 2 years. Cognitive decline was defined by calculating the Reliable Change Index using the Mini-Mental State Examination. Physical activity levels were categorized as sedentary, low, or high. Changes in physical activity were classified as inactive, decreaser, increaser, or active. Logistic regression analysis of baseline and changes in physical activity with cognitive decline was performed. Compared with the sedentary group at baseline, both the low and high activity groups were less likely to experience cognitive decline. The active (odds ratio [OR] = 0.40, 95 % confidence interval [CI] 0.23-0.68) and increaser (OR = 0.45, 95 % CI 0.27-0.74) group, compared with the inactive counterpart, demonstrated a significantly lower likelihood of cognitive decline. Older adults who remained active or increased activity over time had a reduced risk of cognitive decline. Engagement in physical activity in late life may have cognitive health benefits.