Identifying the Essential Portions of the Skill Acquisition Process Using Item Response Theory

Abstract

Objective: Item response theory (IRT) was originally developed to make performance assessments more accurate. However, IRT analysis of the intraoperative performance of surgical trainees could help identify the elements that the trainees find difficult during the skill acquisition process. The aim of this study was to identify the essential portions of the skill acquisition process of a surgical procedure using the IRT.

Design: The 24-item assessment checklist was used to evaluate a recorded intra-operative performance of a laparoscopic inguinal hernia repair. The scores were analyzed using IRT to calculate the difficulty and discrimination level of each item.

Setting: Fifteen institutes.

Participants: Thirty surgical trainees.

Results: A total of 123 assessments were analyzed. The item analysis showed the procedure specific item "traction of peritoneum (difficulty: -0.45; discrimination: 19.37)" and generic items "instrument handling (difficulty: -0.59; discrimination: 3.82)" and "flow of procedure (difficulty: 0.09; discrimination: 3.27)" to be key elements in the skill acquisition process of the procedure.

Conclusions: Key elements in the skill acquisition process of the procedure were quantitatively identified by applying the IRT analysis. This could lead to the use of IRT in designing and developing a more effective training curriculum.

Keywords: IRT; Item Response Theory; Patient Care; Practice-Based Learning and Improvement; Systems-Based Practice; TAPP; item response theory analysis; laparoscopic inguinal hernia repair; skill acquisition; transabdominal preperitoneal.