Objective: To evaluate the performance of an internally developed and previously validated artificial intelligence (AI) algorithm for magnetic resonance (MR)-derived total kidney volume (TKV) in autosomal dominant polycystic kidney disease (ADPKD) when implemented in clinical practice.
Patients and methods: The study included adult patients with ADPKD seen by a nephrologist at our institution between November 2019 and January 2021 and undergoing an MR imaging examination as part of standard clinical care. Thirty-three nephrologists ordered MR imaging, requesting AI-based TKV calculation for 170 cases in these 161 unique patients. We tracked implementation and performance of the algorithm over 1 year. A radiologist and a radiology technologist reviewed all cases (N=170) for quality and accuracy. Manual editing of algorithm output occurred at radiology or radiology technologist discretion. Performance was assessed by comparing AI-based and manually edited segmentations via measures of similarity and dissimilarity to ensure expected performance. We analyzed ADPKD severity class assignment of algorithm-derived vs manually edited TKV to assess impact.
Results: Clinical implementation was successful. Artificial intelligence algorithm-based segmentation showed high levels of agreement and was noninferior to interobserver variability and other methods for determining TKV. Of manually edited cases (n=84), the AI-algorithm TKV output showed a small mean volume difference of -3.3%. Agreement for disease class between AI-based and manually edited segmentation was high (five cases differed).
Conclusion: Performance of an AI algorithm in real-life clinical practice can be preserved if there is careful development and validation and if the implementation environment closely matches the development conditions.
Copyright © 2023 The Authors. Published by Elsevier Inc. All rights reserved.