Surgeon-performed ultrasonography

Abstract

Surgeons are increasingly using ultrasonography (US) in their clinical management of patients. However, US is a very user-dependent imaging modality and proper skills of the US operator are needed to ensure quality in patient care. This thesis explores the validity evidence for assessment of competence in abdominal and head & neck ultrasonography using the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale. With the use of Messick's unitary framework of validity, five sources of validity evidence were explored: test content, response processes, inter-nal structure, relations to other variables, and consequences. Research paper I examined validity evidence for the use of the OSAUS scale to assess physicians' abdominal point-of-care US competence in an experimental setting using patient cases with and without pathological conditions. The RESULTS provided validity evidence of the internal structure of the OSAUS scale and a deci-sion study predicted that four cases and two raters or five cases and one rater could ensure sufficient reliability in future test setups. The relation to other variables was supported by a signifi-cant difference in scores between US experience levels, and by a strong correlation between the OSAUS score and diagnostic accuracy. Research paper II explored the transfer of learning from formal point-of-care US training to performance on patients in a randomized controlled study. The RESULTS supported validity evi-dence regarding OSAUS scores' relation to other variables by demonstrating a significant discrimination in the progress of training-a more refined validity evidence than the relation to difference experience levels. The RESULTS showed that physicians could transfer the skills learned on an ultrasonography course to improved US performance and diagnostic accuracy on patients. However, the RESULTS also indicated that following an initial course, additional training is needed for physicians to achieve competence in US. Research paper III evaluated validity evidence supporting an OSAUS score used to establish pass/fail standards for head & neck US skills. Good reliability between raters from different specialties to assess head & neck competence further supported the internal structure of OSAUS scale. A strong correla-tion to the diagnostic accuracy supported the relation to other variables and the consequences of the assessment were explored by a receiver operator characteristic curve for different pass/fail standards of head & neck US skills. In summary this PhD thesis established sources of validity evi-dence supporting the interpretation of the OSAUS scale to evalu-ate surgeon-performed US skills of the abdominal and head & neck diseases. We therefore recommend the OSAUS scale for formative in-training assessment and high-stakes summative decisions as certification for independent practice in surgeon-performed US. Further, we find formal "hands on" courses an essential part of initial US training with good transfer of learning to improved diagnostic accuracy. This thesis can therefore be used to support the move towards competency-based training in abdominal and head & neck US.