Both high and low vitamin D statuses have been associated with lower memory function. Apolipoprotein E (APOE) ɛ4 alleles have been associated with reduced memory function, and separately with higher vitamin D concentrations. This report aims to examine if the presence of APOE ɛ4 alleles contributes to the relationship between vitamin D and memory function. A total of 4848 (46% female) participants from the 1958 British birth cohort had information on APOE genotypes and completed memory tests at 50 years, where 4644 also had 25-hydroxyvitamin D (25(OH)D) concentrations measured at 45 years. Both low and high 25(OH)D concentrations were associated with lower memory function after adjustment for number of APOE ɛ4 alleles (P curvature=0.02). There was evidence of interaction between APOE ɛ4 and 25(OH)D, suggesting the association between 25(OH)D concentrations and memory function is different for those with two APOE ɛ4 alleles compared with those with zero or one APOE ɛ4 alleles (recessive model P interaction=0.01). Among participants with two APOE ɛ4 alleles, higher 25(OH)D concentrations were associated with higher memory function, whereas in others, memory scores were slightly lower for individuals with higher versus lower concentrations. Further studies are required to replicate these findings.