Objectives: The main objectives of this study are to identify the dimensions of Ecological Validity (EV) within the definitions of this concept, understand how they are operationalized in neurocognitive tests, and propose a checklist for EV attributes in neurocognitive tests.
Method: A systematized review was combined with content analysis of the selected papers, using the inductive method. We analyzed 82 studies on the EV of neurocognitive tests, 19 literature reviews and 63 empirical studies. Based on this review, we identified the relevant criteria for evaluating EV.
Results: EV is a multidimensional concept with two main dimensions: representativeness and generalization. Representativeness involves the subdimensions simplicity-complexity and artificial-natural and several criteria organized on a continuum from low EV to high EV. Generalization is dependent on representativeness and is influenced by different cognitive and non-cognitive factors. We propose six stages for operationalizing EV, from defining the objectives of the neurocognitive assessment to the methodology for scoring and interpreting the results.
Conclusion: This systematized review helps to operationalize the concept of EV by providing a tool for evaluating and improving EV while developing new tests. Further studies with a longitudinal design can compare the predictive value of tests with higher versus lower EV-checklist scores.Key pointsQuestion: Understand the definition of EV, its dimensions and subdimensions, how EV is operationalized in neurocognitive tests and propose a checklist for the EV attributes of neurocognitive tests.Findings: The primary findings were that representativeness and generalization are the main dimensions of EV. Representativeness involves several subdimensions, whereas generalization is dependent on representativeness and is influenced by cognitive and non-cognitive factors. We provided an EV-checklist organized into six parts.Importance: The EV-checklist can be used to guide the development of ecologically valid neurocognitive tests and/or assess the EV of existing ones.Next steps: Examine the predictive value of tests that have higher EV-checklist scores.
Keywords: Checklist; content analysis; ecological validity; neurocognitive assessment; systematized review.