The three ghosts of medical AI: Can the black-box present deliver?

Thomas P Quinn; Stephan Jacobs; Manisha Senadeera; Vuong Le; Simon Coghlan

doi:10.1016/j.artmed.2021.102158

The three ghosts of medical AI: Can the black-box present deliver?

Artif Intell Med. 2022 Feb:124:102158. doi: 10.1016/j.artmed.2021.102158. Epub 2021 Aug 28.

Authors

Thomas P Quinn¹, Stephan Jacobs², Manisha Senadeera², Vuong Le², Simon Coghlan³

Affiliations

¹ Applied Artificial Intelligence Institute, Deakin University, Geelong, Australia. Electronic address: contacttomquinn@gmail.com.
² Applied Artificial Intelligence Institute, Deakin University, Geelong, Australia.
³ Centre for AI and Digital Ethics, School of Computing and Information Systems, The University of Melbourne, Melbourne, Australia.

PMID: 34511267
DOI: 10.1016/j.artmed.2021.102158

Abstract

Our title alludes to the three Christmas ghosts encountered by Ebenezer Scrooge in A Christmas Carol, who guide Ebenezer through the past, present, and future of Christmas holiday events. Similarly, our article takes readers through a journey of the past, present, and future of medical AI. In doing so, we focus on the crux of modern machine learning: the reliance on powerful but intrinsically opaque models. When applied to the healthcare domain, these models fail to meet the needs for transparency that their clinician and patient end-users require. We review the implications of this failure, and argue that opaque models (1) lack quality assurance, (2) fail to elicit trust, and (3) restrict physician-patient dialogue. We then discuss how upholding transparency in all aspects of model design and model validation can help ensure the reliability and success of medical AI.

Keywords: Autonomy; Black-box; Challenges; Ethics; Transparency; Xai.

Publication types

Review

MeSH terms

Artificial Intelligence*
Delivery of Health Care
Humans
Machine Learning*
Reproducibility of Results
Trust