Population data-based federated machine learning improves automated echocardiographic quantification of cardiac structure and function: the Automatisierte Vermessung der Echokardiographie project

Caroline Morbach; Götz Gelbrich; Marcus Schreckenberg; Maike Hedemann; Dora Pelin; Nina Scholz; Olga Miljukov; Achim Wagner; Fabian Theisen; Niklas Hitschrich; Hendrik Wiebel; Daniel Stapf; Oliver Karch; Stefan Frantz; Peter U Heuschmann; Stefan Störk

doi:10.1093/ehjdh/ztad069

Population data-based federated machine learning improves automated echocardiographic quantification of cardiac structure and function: the Automatisierte Vermessung der Echokardiographie project

Eur Heart J Digit Health. 2023 Nov 15;5(1):77-88. doi: 10.1093/ehjdh/ztad069. eCollection 2024 Jan.

Authors

Caroline Morbach^{1

2}, Götz Gelbrich^{3

4}, Marcus Schreckenberg⁵, Maike Hedemann¹, Dora Pelin¹, Nina Scholz¹, Olga Miljukov³, Achim Wagner⁶, Fabian Theisen¹, Niklas Hitschrich⁵, Hendrik Wiebel⁵, Daniel Stapf⁵, Oliver Karch⁶, Stefan Frantz^{1

2}, Peter U Heuschmann^{3

4}, Stefan Störk^{1

2}

Affiliations

¹ Department Clinical Research and Epidemiology, Comprehensive Heart Failure Center, University Hospital Würzburg, Am Schwarzenberg 15, D-97078 Würzburg, Germany.
² Department of Medicine I, University Hospital Würzburg, Oberdürrbacherstr. 6, D-97080 Würzburg, Germany.
³ Institute of Clinical Epidemiology and Biometry, University of Würzburg, Josef-Schneider-Str. 2, 97080 Würzburg, Germany.
⁴ Clinical Trial Center, University Hospital Würzburg, Josef-Schneider-Str. 2, 97080 Würzburg, Germany.
⁵ TOMTEC Imaging Systems GmbH, Freisinger Str. 9, 85716 Unterschleissheim, Germany.
⁶ Service Center Medical Informatics, University Hospital Würzburg, Josef-Schneider-Str. 2, 97080 Würzburg, Germany.

Abstract

Aims: Machine-learning (ML)-based automated measurement of echocardiography images emerges as an option to reduce observer variability. The objective of the study is to improve the accuracy of a pre-existing automated reading tool ('original detector') by federated ML-based re-training.

Methods and results: Automatisierte Vermessung der Echokardiographie was based on the echocardiography images of n = 4965 participants of the population-based Characteristics and Course of Heart Failure Stages A-B and Determinants of Progression Cohort Study. We implemented federated ML: echocardiography images were read by the Academic Core Lab Ultrasound-based Cardiovascular Imaging at the University Hospital Würzburg (UKW). A random algorithm selected 3226 participants for re-training of the original detector. According to data protection rules, the generation of ground truth and ML training cycles took place within the UKW network. Only non-personal training weights were exchanged with the external cooperation partner for the refinement of ML algorithms. Both the original detectors as the re-trained detector were then applied to the echocardiograms of n = 563 participants not used for training. With regard to the human referent, the re-trained detector revealed (i) superior accuracy when contrasted with the original detector's performance as it arrived at significantly smaller mean differences in all but one parameter, and a (ii) smaller absolute difference between measurements when compared with a group of different human observers.

Conclusion: Population data-based ML in a federated ML set-up was feasible. The re-trained detector exhibited a much lower measurement variability than human readers. This gain in accuracy and precision strengthens the confidence in automated echocardiographic readings, which carries large potential for applications in various settings.

Keywords: Automated measurement; Machine learning; Measurement variability; Observer variability; Population-based cohort; Sample size.