Validity and Reliability of Executive Function Measures in Children With Heavy Prenatal Alcohol Exposure: Correspondence Between Multiple Raters and Laboratory Measures

Gemma A Bernes; Miguel Villodas; Claire D Coles; Julie A Kable; Philip A May; Wendy O Kalberg; Elizabeth R Sowell; Kenneth L Jones; Edward P Riley; Sarah N Mattson; CIFASD

doi:10.1111/acer.14547

Validity and Reliability of Executive Function Measures in Children With Heavy Prenatal Alcohol Exposure: Correspondence Between Multiple Raters and Laboratory Measures

Alcohol Clin Exp Res. 2021 Mar;45(3):596-607. doi: 10.1111/acer.14547. Epub 2021 Feb 15.

Authors

Gemma A Bernes^{1

2}, Miguel Villodas², Claire D Coles^{3

4}, Julie A Kable^{3

4}, Philip A May^{5

6}, Wendy O Kalberg⁶, Elizabeth R Sowell⁷, Kenneth L Jones⁸, Edward P Riley^{1

2}, Sarah N Mattson^{1

2}; CIFASD

Affiliations

¹ Center for Behavioral Teratology, San Diego State University, San Diego, CA, USA.
² Department of Psychology, San Diego State University, San Diego, CA, USA.
³ Department of Psychiatry and Behavioral Sciences, Emory University School of Medicine, Atlanta, GA, USA.
⁴ Department of Pediatrics, Emory University School of Medicine, Atlanta, GA, USA.
⁵ Department of Nutrition, Gillings School of Global Public Health, University of North Carolina Nutrition Research Institute, Kannapolis, NC, USA.
⁶ Center on Alcoholism, Substance Abuse, and Addictions, University of New Mexico, Albuquerque, NM, USA.
⁷ Department of Pediatrics, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.
⁸ Department of Pediatrics, University of California, San Diego School of Medicine, San Diego, CA, USA.

Abstract

Background: Rating scales are designed to complement traditional performance-based measures, and both can provide useful information about the functioning of youth with histories of prenatal alcohol exposure. Few studies, however, have compared ratings from multiple informants or the relationship between these subjective rating scale scores and the objective results from laboratory performance-based scales.

Methods: The current study addressed both of these questions in 3 study groups: children with histories of prenatal alcohol exposure (n = 47), attention-deficit/hyperactivity disorder (ADHD; n = 41), and typically developing controls (CON; n = 73). All subjects completed a standardized neuropsychological test battery, including laboratory measures of executive functioning and a self-report measure of executive function behaviors. Parents and teachers completed corresponding rating scales of executive function behaviors for each subject. This study assessed the relationship between these behavior rating scales and corresponding neuropsychological tests, and interrater agreement among the multiple informants.

Results: Weak correlations were found between the rating scales and laboratory measures, indicating poor convergent validity for the behavior rating scale. Interrater reliability was found but it differed by group. Agreement was found between parent and teacher ratings for children with prenatal alcohol exposure, whereas teacher-child agreement was found for those with ADHD.

Conclusions: Findings from this study indicate that behavior ratings can be used to supplement laboratory measures but may not be measuring cognitive abilities regardless of whether a clinical diagnosis is present. A multimethod approach should be used when measuring skills in this domain. This was one of the first studies to examine cross-informant agreement in a sample of children with prenatal alcohol exposure. Further research is necessary to understand why interrater agreement differed for children with prenatal alcohol exposure and those with ADHD.

Keywords: Behavior Rating Scales; Executive Function; Fetal Alcohol Spectrum Disorders; Interrater Agreement; Validity.

Publication types

Multicenter Study
Research Support, N.I.H., Extramural

MeSH terms

Adolescent
Alcohol Drinking / adverse effects*
Alcohol Drinking / psychology
Attention Deficit Disorder with Hyperactivity
Behavior Rating Scale / standards*
Child
Clinical Laboratory Techniques / standards*
Executive Function / physiology*
Female
Fetal Alcohol Spectrum Disorders / diagnosis*
Fetal Alcohol Spectrum Disorders / psychology
Humans
Male
Neuropsychological Tests / standards*
Pregnancy
Prenatal Exposure Delayed Effects / diagnosis
Prenatal Exposure Delayed Effects / psychology
Reproducibility of Results

Abstract

Publication types

MeSH terms

Grants and funding