OBJECTIVES: The aim of this study was to examine the longitudinal association between seafood and intake of long-chain ω-3 polyunsaturated fatty acids (LCω-3 PUFA) and cognitive function and to explore the possible effect modifications owing to mercury (Hg) and selenium (Se) levels. METHODS: Participants (N = 3231) from the CARDIA (Coronary Artery Risk Development in Young Adults) study underwent baseline examination and were reexamined in eight follow-up visits. Diet was assessed at baseline and in exam years 7 and 20. Toenail Hg and Se were measured at exam year 2. Cognitive function was measured at exam year 25 using three tests: Rey Auditory Verbal Learning Test (RAVLT), Digit Symbol Substitution Test (DSST), and the Stroop test. The general linear regression model was used to examine cumulative average intakes of LCω-3 PUFA and seafood in relation to the cognitive test scores; and to explore the possible effect modifications caused by Hg and Se. RESULTS: LCω-3 PUFA intake was significantly associated with better performance in the DSST test (quintile 5 versus quintile 1; mean difference = 1.74; 95% confidence interval, 0.19-3.29; Ptrend, 0.048]), but not in the RAVLT and Stroop tests. Similar results were observed for intakes of eicosapentaenoic acid, docosahexaenoic acid, and non-fried seafood. The observed associations were more pronounced in participants with body mass index ≥25 kg/m$^2$, but not significantly modified by toenail Hg or Se. CONCLUSION: This longitudinal study supported the hypothesis that LCω-3 PUFA or non-fried seafood intake is associated with better cognitive performance in psychomotor speed among US adults, especially those who are overweight or obese.