Concurrent and predictive validation of a novel robotic surgery simulator: a prospective, randomized study

Abstract

Purpose: We evaluated the concurrent and predictive validity of a novel robotic surgery simulator in a prospective, randomized study.

Materials and methods: A total of 24 robotic surgery trainees performed virtual reality exercises on the da Vinci® Skills Simulator using the da Vinci Si™ surgeon console. Baseline simulator performance was captured. Baseline live robotic performance on ex vivo animal tissue exercises was evaluated by 3 expert robotic surgeons using validated laparoscopic assessment metrics. Trainees were then randomized to group 1-simulator training and group 2-no training while matched for baseline tissue scores. Group 1 trainees underwent a 10-week simulator curriculum. Repeat tissue exercises were done at study conclusion to assess performance improvement. Spearman's analysis was used to correlate baseline simulator performance with baseline ex vivo tissue performance (concurrent validity) and final tissue performance (predictive validity). The Kruskal-Wallis test was used to compare group performance.

Results: Groups 1 and 2 were comparable in pre-study surgical experience and had similar baseline scores on simulator and tissue exercises (p >0.05). Overall baseline simulator performance significantly correlated with baseline and final tissue performance (concurrent and predictive validity each r = 0.7, p <0.0001). Simulator training significantly improved tissue performance on key metrics for group 1 subjects with lower baseline tissue scores (below the 50th percentile) than their group 2 counterparts (p <0.05). Group 1 tended to outperform group 2 on final tissue performance, although the difference was not significant (p >0.05).

Conclusions: Our study documents the concurrent and predictive validity of the Skills Simulator. The benefit of simulator training appears to be most substantial for trainees with low baseline robotic skills.