Purpose: There is a growing interest in situational judgment tests (SJTs) for assessments and the authors offer their insights to help others develop and utilize these instruments while navigating common challenges.
Description: This manuscript provides important considerations to those interested in SJTs using the "What? So What? Now What?" reflection framework. The authors offer their experience developing, implementing, and evaluating SJTs designed to measure empathy, adaptability, and integrity.
Analysis/interpretation: Effective SJT design can be organized around five key steps: attribute definition, item design, key development and scoring, test administration, and data analysis and interpretation. There are general recommendations to optimize SJT design to enhance reliability and validity of the results; however, there are often other challenges that have not been previously described.
Conclusions: SJTs offer an exciting tool for measuring social and behavioral attributes; however, their infancy in health professions education warrants caution. Like any assessment, there are several challenges to consider when designing, implementing, and evaluating SJTs in pharmacy education.
Implications: Individuals interested in SJTs are encouraged to ensure the construct measured is well-defined, there are clear and supported decisions around item and key design, and the appropriate psychometric analyses are conducted.
Keywords: Assessment; Reliability; Situational judgment test; Validity.
Copyright © 2022 Elsevier Inc. All rights reserved.