We evaluated six estrogen receptor 1 (ESR1) polymorphisms for association with ten plasma lipid and apolipoprotein traits in 1,847 individuals (941 females and 906 males) in the multi-generation Rochester Family Heart Study using a generalized estimating equation approach. Apolipoprotein A-I (apoA-I), apoA-II, and HDL-cholesterol (HDL-C) were associated with exon 4 rs1801132 (Pro325Pro) genotype (P = 0.0044, P = 0.0048, and P = 0.0035, respectively). Positive correlation between levels of apoA-I, apoA-II, and HDL-C and the number of G alleles was observed in females (P = 0.0120, P = 0.0032, and P = 0.0030), but not males (P > 0.05). Because few studies have evaluated the effect of ESR1 gene polymorphisms on lipid traits in children, we also stratified our sample at the age of 15 years. There was evidence of association between intron 1 single-nucleotide polymorphisms rs9322331 and rs9340799 and apoC-II, and triglycerides (TGs) in youths 15 years and younger. In youths, evidence of association between rs9322331 and rs9340799 and apoC-II was stronger in males (P = 0.0036 and P = 0.0124) than in females (P > 0.05), whereas evidence of association with TG was stronger in females (P = 0.0030 and P = 0.0024) than in males (P > 0.05). These findings suggest that ESR1 variation plays an age- and sex-dependent role in determining plasma lipid and apolipoprotein levels.