An Evaluation of Computational Learning-based Methods for the Segmentation of Nuclei in Cervical Cancer Cells from Microscopic Images

Abstract

Background: The manual segmentation of cellular structures on Z-stack microscopic images is time-consuming and often inaccurate, highlighting the need to develop auto-segmentation tools to facilitate this process.

Objective: This study aimed to compare the performance of three different machine learning architectures, including random forest (RF), AdaBoost, and multi-layer perceptron (MLP), for the autosegmentation of nuclei in proliferating cervical cancer cells on Z-Stack cellular microscopy proliferation images provided by the HCS Pharma. The impact of using post-processing techniques, such as the StarDist plugin and majority voting, was also evaluated.

Methods: The RF, AdaBoost, and MLP algorithms were used to auto-segment the nuclei of cervical cancer cells on microscopic images at different Z-stack positions. Post-processing techniques were then applied to each algorithm. The performance of all algorithms was compared by an expert to globally generated ground truth by calculating the accuracy detection rate, the Dice coefficient, and the Jaccard index.

Results: RF achieved the best accuracy, followed by the AdaBoost and then the MLP. All algorithms achieved good pixel classifications except in regions whereby the nuclei overlapped. The majority voting and StarDist plugin improved the accuracy of the segmentation but did not resolve the nuclei overlap issue. The Z-Stack analysis revealed similar segmentation results to the Z-stack layer used to train the image. However, a worse performance was noted for segmentations performed on different Z-stack positions, which were not used to train the algorithms.

Conclusion: All machine learning architectures provided a good segmentation of nuclei in cervical cancer cells but did not resolve the problem of overlapping nuclei and Z-stack segmentation. Further research should therefore evaluate the combined segmentation techniques and deep learning architectures to resolve these issues.

Keywords: BIOMIMESYS; High content screening; Z-Stack; machine learning; majority voting; metrics; segmentation.