Reliability and Variability of Ki-67 Digital Image Analysis Methods for Clinical Diagnostics in Breast Cancer

Melanie Dawe; Wei Shi; Tian Y Liu; Katherine Lajkosz; Yukiko Shibahara; Nakita E K Gopal; Rokshana Geread; Seyed Mirjahanmardi; Carrie X Wei; Sehrish Butt; Moustafa Abdalla; Sabrina Manolescu; Sheng-Ben Liang; Dianne Chadwick; Michael H A Roehrl; Trevor D McKee; Adewunmi Adeoye; David McCready; April Khademi; Fei-Fei Liu; Anthony Fyles; Susan J Done

doi:10.1016/j.labinv.2024.100341

Reliability and Variability of Ki-67 Digital Image Analysis Methods for Clinical Diagnostics in Breast Cancer

Lab Invest. 2024 Jan 25;104(5):100341. doi: 10.1016/j.labinv.2024.100341. Online ahead of print.

Authors

Melanie Dawe¹, Wei Shi¹, Tian Y Liu², Katherine Lajkosz¹, Yukiko Shibahara³, Nakita E K Gopal⁴, Rokshana Geread⁵, Seyed Mirjahanmardi⁶, Carrie X Wei¹, Sehrish Butt⁷, Moustafa Abdalla¹, Sabrina Manolescu¹, Sheng-Ben Liang⁸, Dianne Chadwick⁹, Michael H A Roehrl¹⁰, Trevor D McKee⁷, Adewunmi Adeoye¹, David McCready¹, April Khademi¹¹, Fei-Fei Liu¹, Anthony Fyles¹, Susan J Done¹²

Affiliations

¹ Princess Margaret Cancer Centre, University Health Network, Toronto, Canada.
² Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Canada.
³ Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Canada; Laboratory Medicine Program, University Health Network, Toronto, Canada.
⁴ Princess Margaret Cancer Centre, University Health Network, Toronto, Canada; Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Canada.
⁵ Department of Electrical, Computer and Biomedical Engineering, Toronto Metropolitan University, Toronto, Canada.
⁶ Department of Electrical, Computer and Biomedical Engineering, Toronto Metropolitan University, Toronto, Canada; Division of Medical Physics, Department of Radiation Oncology, Stanford University, Stanford, California.
⁷ STTARR Innovation Centre, University Health Network, Toronto, Canada.
⁸ Princess Margaret Cancer Biobank, University Health Network, Toronto, Canada.
⁹ Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Canada; Laboratory Medicine Program, University Health Network, Toronto, Canada; Princess Margaret Cancer Biobank, University Health Network, Toronto, Canada; Ontario Tumour Bank, Ontario Institute for Cancer Research, Toronto, Canada.
¹⁰ Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Canada; Laboratory Medicine Program, University Health Network, Toronto, Canada; Princess Margaret Cancer Biobank, University Health Network, Toronto, Canada; Department of Pathology, Beth Israel Deaconess Medical Center, Boston, Massachusetts.
¹¹ Department of Electrical, Computer and Biomedical Engineering, Toronto Metropolitan University, Toronto, Canada; St Michael's Hospital, Unity Health Network, Toronto, Canada.
¹² Princess Margaret Cancer Centre, University Health Network, Toronto, Canada; Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Canada; Laboratory Medicine Program, University Health Network, Toronto, Canada. Electronic address: susan.done@uhn.ca.

PMID: 38280634
DOI: 10.1016/j.labinv.2024.100341

Abstract

Ki-67 is a nuclear protein associated with proliferation, and a strong potential biomarker in breast cancer, but is not routinely measured in current clinical management owing to a lack of standardization. Digital image analysis (DIA) is a promising technology that could allow high-throughput analysis and standardization. There is a dearth of data on the clinical reliability as well as intra- and interalgorithmic variability of different DIA methods. In this study, we scored and compared a set of breast cancer cases in which manually counted Ki-67 has already been demonstrated to have prognostic value (n = 278) to 5 DIA methods, namely Aperio ePathology (Lieca Biosystems), Definiens Tissue Studio (Definiens AG), Qupath, an unsupervised immunohistochemical color histogram algorithm, and a deep-learning pipeline piNET. The piNET system achieved high agreement (interclass correlation coefficient: 0.850) and correlation (R = 0.85) with the reference score. The Qupath algorithm exhibited a high degree of reproducibility among all rater instances (interclass correlation coefficient: 0.889). Although piNET performed well against absolute manual counts, none of the tested DIA methods classified common Ki-67 cutoffs with high agreement or reached the clinically relevant Cohen's κ of at least 0.8. The highest agreement achieved was a Cohen's κ statistic of 0.73 for cutoffs 20% and 25% by the piNET system. The main contributors to interalgorithmic variation and poor cutoff characterization included heterogeneous tumor biology, varying algorithm implementation, and setting assignments. It appears that image segmentation is the primary explanation for semiautomated intra-algorithmic variation, which involves significant manual intervention to correct. Automated pipelines, such as piNET, may be crucial in developing robust and reproducible unbiased DIA approaches to accurately quantify Ki-67 for clinical diagnosis in the future.

Keywords: Ki-67; biomarker; breast cancer; digital pathology; image analysis.