Development and Validation of a Point-of-Care-Ultrasound Image Quality Assessment Tool: The POCUS IQ Scale

Abstract

Objectives: We aimed to develop a standardized scoring tool to measure point-of-care ultrasound (POCUS) image quality and to determine validity evidence for its use to assess lung ultrasound image quality.

Methods: The POCUS Image Quality (POCUS IQ) scale was developed by POCUS-trained physicians to assess sonographers' image acquisition skills by evaluating image quality for any POCUS application. The scale was piloted using lung images of healthy standardized patients acquired by three expert sonographers compared to three novices before and after training. All images (experts, novices pre-training, novices post-training) were scored on the POCUS IQ scale by three blinded POCUS-trained physicians. Reliability was assessed with fully-crossed generalizability and decision studies. Validity was assessed using Messick's framework.

Results: Content validity was supported by the tool's development process of literature review, expert consensus, and pilot testing. Response process was supported by reviewer training and the blinded scoring process. Relation to other variables was supported by scores relating to sonographer experience: median expert score = 10.5/14 (IQR: 4), median novice pre-training score = 6/14 (IQR: 2.25), and novices' improvement after training (median post-training score = 12/14, IQR: 3.25). Internal structure was supported by internal consistency data (coefficient alpha = 0.84, omega coefficient = 0.91) and the generalizability study showing the main contributor to score variability was the sonographer (51%). The G-coefficient was 0.89, suggesting very good internal structure, however, Gwet's AC₂ was 0.5, indicating moderate interrater reliability. The D study projected a minimum of 1 reviewer and 2 patients are needed for good psychometric reliability.

Conclusions: The POCUS scale has good preliminary validity evidence as an assessment tool for lung POCUS image acquisition skills. Further studies are needed to demonstrate its utility for other POCUS applications and as a feedback tool for POCUS learners.

Keywords: G study; bedside ultrasound; clinical ultrasound; competency assessment; generalizability theory; point-of-care ultrasound.