A standardized measurement instrument was recommended for evaluating operator experience in complex healthcare interventions

Abstract

Objective: During development of complex surgical innovations, modifications occur to optimize safety and efficacy. Operators' experiences (how professionals feel undertaking the innovation) drive this process but comprehensive overviews of measures of this concept are lacking. This study identified and appraised measures to assess operators' experience of surgical innovation.

Study design and setting: There were three phases: (1) Literature reviews identified measures of operators' experience and concepts measured were extracted and grouped into domains. (2) Quality appraisal was conducted to assess content validity of identified instruments and was supported by COnsensus-based Standards for the selection of health Measurement Instruments methodology. Self-reported measurement instruments that had underdone formal development were eligible. Content validity was assessed using COnsensus-based Standards for the selection of health Measurement Instruments criteria for good content validity (rated sufficient/insufficient/indeterminate/inconsistent), informed by standards for measurement development and domains identified in phase 1. (3) Instruments determined suitable and of sufficient quality underwent supplemental appraisal in interviews with international multidisciplinary professionals and a focus group.

Results: Literature reviews identified 16 measurement instruments from 243 studies. Most assessed 'psychological' experiences and 'usability'. No instrument was specifically validated for innovative surgery. Three instruments were rated 'sufficient' (Surgery Task Load Index [SURG-TLX]) or 'indeterminate' (Spielberger State-Trait Anxiety Inventory, Imperial Stress Assessment Tool). Twenty professionals were interviewed (seven female; 15 specialties; six countries) and the focus group included 10 participants (four professionals, six researchers). The SURG-TLX was considered the most relevant, comprehensive, and comprehensible instrument.

Conclusion: The SURG-TLX is preliminarily recommended to measure operators' experiences of innovation. Further work exploring its role and impact on surgical innovation is required.

Keywords: COSMIN; Complex healthcare intervention; Measurement instrument; Operator experience; Outcome; Outcome measurement; SURG-TLX; Surgery; Surgical innovation.