Multifactorial lifestyle intervention is known to be more effective for ameliorating cognitive decline than single factor intervention; however, the effects of combining exercise with long-chain polyunsaturated fatty acids (LCPUFA) on the elderlies' cognitive function remain unclear. We conducted a randomised, single-masked placebo-controlled trial in non-demented elderly Japanese individuals. Participants were randomly allocated to the exercise with LCPUFA, placebo, or no exercise with placebo (control) groups. Participants in the exercise groups performed 150 min of exercise per week, comprised resistance and aerobic training, for 24 weeks with supplements of either LCPUFA (docosahexaenoic acid, 300 mg/day; eicosapentaenoic acid, 100 mg/day; arachidonic acid, 120 mg/day) or placebo. Cognitive functions were evaluated by neuropsychological tests prior to and following the intervention. The per-protocol set analysis (n = 76) revealed no significant differences between the exercise and the control groups in changes of neuropsychological tests. Subgroup analysis for participants with low skeletal muscle mass index (SMI) corresponding to sarcopenia cut-off value showed changes in selective attention, while working memory in the exercise with LCPUFA group was better than in the control group. These findings suggest that exercise with LCPUFA supplementation potentially improves attention and working memory in the elderly with low SMI.