Predicting stone composition via machine-learning models trained on intra-operative endoscopic digital images

Abstract

Objectives: The aim of this study was to use deep learning (DL) of intraoperative images of urinary stones to predict the composition of urinary stones. In this way, the laser frequency and intensity can be adjusted in real time to reduce operation time and surgical trauma.

Materials and methods: A total of 490 patients who underwent holmium laser surgery during the two-year period from March 2021 to March 2023 and had stone analysis results were collected by the stone laboratory. A total of 1658 intraoperative stone images were obtained. The eight stone categories with the highest number of stones were selected by sorting. Single component stones include calcium oxalate monohydrate (W1), calcium oxalate dihydrate (W2), magnesium ammonium phosphate hexahydrate, apatite carbonate (CH) and anhydrous uric acid (U). Mixed stones include W2 + U, W1 + W2 and W1 + CH. All stones have intraoperative videos. More than 20 intraoperative high-resolution images of the stones, including the surface and core of the stones, were available for each patient via FFmpeg command screenshots. The deep convolutional neural network (CNN) ResNet-101 (ResNet, Microsoft) was applied to each image as a multiclass classification model.

Results: The composition prediction rates for each component were as follows: calcium oxalate monohydrate 99% (n = 142), calcium oxalate dihydrate 100% (n = 29), apatite carbonate 100% (n = 131), anhydrous uric acid 98% (n = 57), W1 + W2 100% (n = 82), W1 + CH 100% ( n = 20) and W2 + U 100% (n = 24). The overall weighted recall of the cellular neural network component analysis for the entire cohort was 99%.

Conclusion: This preliminary study suggests that DL is a promising method for identifying urinary stone components from intraoperative endoscopic images. Compared to intraoperative identification of stone components by the human eye, DL can discriminate single and mixed stone components more accurately and quickly. At the same time, based on the training of stone images in vitro, it is closer to the clinical application of stone images in vivo. This technology can be used to identify the composition of stones in real time and to adjust the frequency and energy intensity of the holmium laser in time. The prediction of stone composition can significantly shorten the operation time, improve the efficiency of stone surgery and prevent the risk of postoperative infection.

Keywords: Automatic recognition; Deep learning; Kidney stones; Morpho-constitutional analysis of urinary stones; Ureteroscopy.