Over-Detection of Melanoma-Suspect Lesions by a CE-Certified Smartphone App: Performance in Comparison to Dermatologists, 2D and 3D Convolutional Neural Networks in a Prospective Data Set of 1204 Pigmented Skin Lesions Involving Patients' Perception

Anna Sophie Jahn; Alexander Andreas Navarini; Sara Elisa Cerminara; Lisa Kostner; Stephanie Marie Huber; Michael Kunz; Julia-Tatjana Maul; Reinhard Dummer; Seraina Sommer; Anja Dominique Neuner; Mitchell Paul Levesque; Phil Fang Cheng; Lara Valeska Maul

doi:10.3390/cancers14153829

Over-Detection of Melanoma-Suspect Lesions by a CE-Certified Smartphone App: Performance in Comparison to Dermatologists, 2D and 3D Convolutional Neural Networks in a Prospective Data Set of 1204 Pigmented Skin Lesions Involving Patients' Perception

Cancers (Basel). 2022 Aug 7;14(15):3829. doi: 10.3390/cancers14153829.

Authors

Affiliations

¹ Department of Dermatology, University Hospital of Basel, 4031 Basel, Switzerland.
² Department of Dermatology, University Hospital of Zurich, 8091 Zurich, Switzerland.
³ Faculty of Medicine, University of Zurich, 8006 Zurich, Switzerland.

Abstract

The exponential increase in algorithm-based mobile health (mHealth) applications (apps) for melanoma screening is a reaction to a growing market. However, the performance of available apps remains to be investigated. In this prospective study, we investigated the diagnostic accuracy of a class 1 CE-certified smartphone app in melanoma risk stratification and its patient and dermatologist satisfaction. Pigmented skin lesions ≥ 3 mm and any suspicious smaller lesions were assessed by the smartphone app SkinVision^® (SkinVision^® B.V., Amsterdam, the Netherlands, App-Version 6.8.1), 2D FotoFinder ATBM^® master (FotoFinder ATBM^® Systems GmbH, Bad Birnbach, Germany, Version 3.3.1.0), 3D Vectra^® WB360 (Canfield Scientific, Parsippany, NJ, USA, Version 4.7.1) total body photography (TBP) devices, and dermatologists. The high-risk score of the smartphone app was compared with the two gold standards: histological diagnosis, or if not available, the combination of dermatologists', 2D and 3D risk assessments. A total of 1204 lesions among 114 patients (mean age 59 years; 51% females (55 patients at high-risk for developing a melanoma, 59 melanoma patients)) were included. The smartphone app's sensitivity, specificity, and area under the receiver operating characteristics (AUROC) varied between 41.3-83.3%, 60.0-82.9%, and 0.62-0.72% according to two study-defined reference standards. Additionally, all patients and dermatologists completed a newly created questionnaire for preference and trust of screening type. The smartphone app was rated as trustworthy by 36% (20/55) of patients at high-risk for melanoma, 49% (29/59) of melanoma patients, and 8.8% (10/114) of dermatologists. Most of the patients rated the 2D TBP imaging (93% (51/55) resp. 88% (52/59)) and the 3D TBP imaging (91% (50/55) resp. 90% (53/59)) as trustworthy. A skin cancer screening by combination of dermatologist and smartphone app was favored by only 1.8% (1/55) resp. 3.4% (2/59) of the patients; no patient preferred an assessment by a smartphone app alone. The diagnostic accuracy in clinical practice was not as reliable as previously advertised and the satisfaction with smartphone apps for melanoma risk stratification was scarce. MHealth apps might be a potential medium to increase awareness for melanoma screening in the lay population, but healthcare professionals and users should be alerted to the potential harm of over-detection and poor performance. In conclusion, we suggest further robust evidence-based evaluation before including market-approved apps in self-examination for public health benefits.

Keywords: diagnostic accuracy; early detection; melanoma; mobile health application; over-detection; smartphone.

Grants and funding

This research project was fully funded by the Department of Dermatology, as well as in minor part by Research Foundation for Young Researchers Grant of the University of Basel, Switzerland, and by the Voluntary Academic Society Grant, Basel, Switzerland. No funding by companies producing the tested products was received, nor did companies have the opportunity to comment or influence the manuscript.