Context: The development of a valid and reliable measure of clinical reasoning ability is a prerequisite to advancing our understanding of clinically relevant cognitive processes and to improving clinical education. A record of problem-solving performances within standardised and computerised patient simulations is often implicitly assumed to reflect clinical reasoning skills. However, the validity of this measurement method for assessing clinical reasoning is open to question.
Objectives: Explicitly defining the intended clinical reasoning construct should help researchers critically evaluate current performance score interpretations. Although case-specific measurement outcomes (i.e. low correlations between cases) have led medical educators to endorse performance-based assessments of problem solving as a method of measuring clinical reasoning, the matter of low across-case generalisation is a reliability issue with validity implications and does not necessarily support a performance-based approach. Given this, it is important to critically examine whether our current performance-based testing efforts are correctly focused. To design a valid educational assessment of clinical reasoning requires a coherent argument represented as a chain of inferences supporting a clinical reasoning interpretation.
Discussion: Suggestions are offered for assessing how well an examinee's existing knowledge organisation accommodates the integration of new patient information, and for focusing assessments on an examinee's understanding of how new patient information changes case-related probabilities and base rates.