Reliability and validity testing of team emergency assessment measure in a distributed team context

Hanna Morian; Maria Härgestam; Magnus Hultin; Håkan Jonsson; Karin Jonsson; Torben Nordahl Amorøe; Johan Creutzfeldt

doi:10.3389/fpsyg.2023.1110306

Reliability and validity testing of team emergency assessment measure in a distributed team context

Front Psychol. 2023 Apr 20:14:1110306. doi: 10.3389/fpsyg.2023.1110306. eCollection 2023.

Authors

Hanna Morian¹, Maria Härgestam¹, Magnus Hultin², Håkan Jonsson³, Karin Jonsson¹, Torben Nordahl Amorøe^{4

5}, Johan Creutzfeldt^{6

7}

Affiliations

¹ Department of Nursing, Umeå University, Umeå, Sweden.
² Department of Surgical and Perioperative Sciences, Anesthesia and Critical Care Medicine, Umeå University, Umeå, Sweden.
³ Department of Epidemiology and Global Health, Umeå University, Umeå, Sweden.
⁴ Department of Molecular and Clinical Medicine, Institute of Medicine, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.
⁵ Simulation Center West, Department of Research, Education, and Development, Region Västra Götaland, Sahlgrenska University Hospital, Gothenburg, Sweden.
⁶ Department of Clinical Science, Intervention, and Technology, Karolinska Institute, Stockholm, Sweden.
⁷ Center for Advanced Medical Simulation and Training, Karolinska University Hospital, Stockholm, Sweden.

Abstract

Medical multi-professional teams are increasingly collaborating via telemedicine. In distributed team settings, members are geographically separated and collaborate through technology. Developing improved training strategies for distributed teams and finding appropriate instruments to assess team performance is necessary. The Team Emergency Assessment Measure (TEAM), an instrument validated in traditional collocated acute-care settings, was tested for validity and reliability in this study when used for distributed teams. Three raters assessed video recordings of simulated team training scenarios (n = 18) among teams with varying levels of proficiency working with a remotely located physician via telemedicine. Inter-rater reliability, determined by intraclass correlation, was 0.74-0.92 on the TEAM instrument's three domains of leadership, teamwork, and task management. Internal consistency (Cronbach's alpha) ranged between 0.89-0.97 for the various domains. Predictive validity was established by comparing scores with proficiency levels. Finally, concurrent validity was established by high correlations, >0.92, between scores in the three TEAM domains and the teams' overall performance. Our results indicate that TEAM can be used in distributed acute-care team settings and consequently applied in future-directed learning and research on distributed healthcare teams.

Keywords: assessments; distributed team; instrument; interprofessional teams; team performance; teamwork; telemedicine; validation.