It is well known that information technology (IT) can play a pivotal role in enhancing healthcare quality and patient safety. The use of computational science to enhance the capabilities of hospital information systems helps deliver enhanced healthcare quality. "Smart healthcare" has become a popular term, reflecting the level of IT involvement in healthcare services. However, each hospital has a different level of IT development, and no clear definition of smart healthcare exists. In this study, we aimed to develop and validate a survey standard to evaluate the level of IT involvement in hospitals. The quality improvement task force of the Joint Commission of Taiwan (QITF-JCT) conducted a systematic literature review to identify the key elements of major healthcare IT functions. The modified Delphi technique was used to review the importance and appropriateness of these elements through an expert panel, and the JCT Smart Healthcare Standard version 1.0 (JCT-SHS 1.0) was drafted. A total of 40 healthcare quality improvement campaign (HQIC) application projects in 2018 were selected for evaluating nine key dimensions of hospital functions: security, digitization, automation, interconnection, connectivity, interoperability, mobility, computation, and artificial intelligence. The standard can be used smart hospital evaluation and executed by two experts by on-site evaluation and rating as three-level scale (norm, excellent, and innovative). The internal consistency and inter-rater reliability were investigated using Cronbach's α and kappa statistics, respectively. This standard was evaluated by using 40 HQIC application projects. The Cronbach's α values were in the range of 0.74-0.92, indicating the good internal consistency of the JCT-SHS 1.0 among the nine IT dimensions. The kappa correlation coefficients were 0.68 for security (p = 0.027), 0.47 for digitization (p = 0.042), 0.21 for automation (p = 0.048), 0.82 for interconnection (p = 0.014), 0.35 for connectivity (p = 0.036), 0.28 for interoperability (p = 0.042), 0.71 for mobility (p = 0.016), 0.47 for computation (p = 0.029), and 0.34 for artificial intelligence (p = 0.033), revealing moderate inter-rater reliability. The validation data indicated that the JCT-SHS 1.0 is a consistent and reliable instrument for evaluating the levels of IT development in the hospitals. Healthcare providers, external accreditation bodies, and policymakers may use the JCT-SHS 1.0 to assess and plan their organizational and system-wise IT strategy.
Keywords: Information technology; Smart healthcare; Standard; Validation.