Assessment of accuracy and reproducibility of cephalometric identification performed by 2 artificial intelligence-driven tracing applications and human examiners

Abstract

Objective: To assess the accuracy and reproducibility of cephalometric landmark identification performed by 2 artificial intelligence (AI)-driven applications (CefBot and WebCeph) and human examiners.

Study design: Lateral cephalometric radiographs of 10 skulls containing 0.5 mm lead spheres directly placed at 10 cephalometric landmarks were obtained as the reference standard. Ten radiographs without spheres were obtained from the same skulls for identification of cephalometric points performed by the AI applications and 10 examiners. The x- and y-coordinate values of the cephalometric points identified by the AI applications and examiners were compared with those from the reference standard images using one-way analysis of variance and the Dunnet post-hoc test. The intraclass correlation coefficient (ICC) was used to evaluate reproducibility. Mean radial error (MRE) in identification was calculated with respect to the reference standard. Statistical significance was established at P < .05.

Results: Landmark identification by CefBot and the examiners did not exhibit significant differences from the reference standard on either axis (P > .05). WebCeph produced a significant difference (P < .05) in 4 and 6 points on the x- and y-axes, respectively. Reproducibility was excellent for CefBot and the examiners (ICC ≥ 0.9943) and good for WebCeph (ICC ≥ 0.7868). MREs of CefBot and the examiners were similar.

Conclusion: With results similar to those of human examiners, CefBot demonstrated excellent reliability and can aid in cephalometric applications. WebCeph produced significant errors.