Usability evaluation of obstetrics and gynecology information system using cognitive walkthrough method

Azadeh Kamel Ghalibaf; Majid Jangi; Mohammad Reza Mazaheri Habibi; Sara Zangouei; Reza Khajouei

doi:10.19082/6682

Usability evaluation of obstetrics and gynecology information system using cognitive walkthrough method

Electron Physician. 2018 Apr 25;10(4):6682-6688. doi: 10.19082/6682. eCollection 2018 Apr.

Authors

Azadeh Kamel Ghalibaf¹, Majid Jangi¹, Mohammad Reza Mazaheri Habibi¹, Sara Zangouei², Reza Khajouei^{3

4}

Affiliations

¹ Ph.D. Candidate, Department of Medical Informatics, Faculty of Medicine, Students Research Committee, Mashhad University of Medical Sciences, Mashhad, Iran.
² M.Sc. of Computer Engineering, Azad University of Mashhad, Mashhad, Iran.
³ Ph.D. in Medical Informatics, Medical Informatics Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran.
⁴ Ph.D. in Medical Informatics, Department of Health Information Sciences, Faculty of Management and Medical Information Sciences, Kerman University of Medical Sciences, Kerman, Iran.

PMID: 29881531
PMCID: PMC5984023
DOI: 10.19082/6682

Abstract

Background and aim: Obstetrics and gynecology information system is a critical component of the HIS in social security organization health centers. The objective of this study was to evaluate the usability of this system using the cognitive walkthrough method. Also, the present study provided a detailed formal description of how the cognitive usability evaluation can be applied and reported for a health care information system.

Methods: This study was conducted at the Mashhad University of Medical Sciences' usability lab from March 2016 to June 2017. A two-phase approach was used to conduct the cognitive walkthrough evaluation: preparatory and evaluation. The preparation was done in three stages: first, we investigated users' capabilities and background knowledge through a semi-structured interview. Second, the evaluation scenario was developed based on the most common tasks in routine workflow of users. Finally, each task was broken down into sequences of actions. In the evaluation phase, three usability experts independently assessed each action using a four-item checklist. Problems were categorized thematically and were reported from three different perspectives: Question-based, Task-based, and Evaluator-based. The data were then analyzed to understand the contribution of each task, along with its mean severity score.

Results: Evaluators' responses were compared and any conflict was resolved in an expert panel. A total of 116 usability problems were identified based on the consensus of the evaluators. Inadequate system feedback was found to be the main source of 43% of the problems, and resulted in users confusion.

Conclusion: Since the system was evaluated in its pilot implementation phase, there was an opportunity to prevent future potential usability problems. The use of a mixed quantitative and qualitative approach in this usability study provided a more comprehensive perspective of the system problems. This study provided a detailed description of conducting CW usability evaluation which can be used as a practical guide for future studies.

Keywords: Cognitive walkthrough; Health information systems; Obstetrics and gynecology; Usability testing.