This study investigated the association between depressive symptoms and cognitive function according to four different trajectories of depressive symptoms in the late middle-aged and older South Korean population. Panel data from the Korean Longitudinal Study of Ageing were analyzed. We used latent class trajectory models to identify four trajectories of depressive symptoms. We performed linear mixed-effects regression analysis to assess associations between depressive-symptom trajectories and MMSE scores. Of 4,374 participants, 18.4%, 4.9%, 55.2%, and 21.5% were classified as having a low, increasing, moderate declining, and high depressive-symptom trajectory, respectively. Individuals with an increasing trajectory (β = -0.729, P ≤ 0.001), moderate trajectory (β = -0.278, P = 0.003), and high trajectory (β = -1.605, P ≤ 0.001) had lower MMSE scores compared with those in the low trajectory group. These relationships were particularly strong among women; individuals who were physically inactive; those who were separated, divorced, or single; and those with hypertension or cerebrovascular disease. Each trajectory group for depressive symptoms was associated with cognitive decline. Moreover, female, physically inactive, and single individuals, as well as those with hypertension and cerebrovascular disease should be particularly mindful of their mental and physical health to prevent cognitive decline.