Automated translation accurately translates recorded pediatric neurosurgery clinic conversations between Spanish and English

Abstract

Objective: The purpose of this study is to analyze an automated voice to text translation device by reporting the translation accuracy for recorded pediatric neurosurgery clinic conversations, classifying errors in translation according to their impact on overall understanding, and comparing the incidence of these errors in English to Spanish vs. Spanish to English conversations.

Methods: English and Spanish speaking patients at a single academic health system's outpatient pediatric neurosurgery clinic had their conversations recorded. These recordings were played back to a Google Pixel handheld smartphone with Live Translate voice to text translation software. A certified medical interpreter evaluated recordings for incidence of minor errors, errors impacting understanding, and catastrophic errors affecting patient-provider relationship or care. Two proportion t-testing was used to compare these outcomes.

Results: 50 patient visits were recorded: 40 English recordings translated to Spanish and 10 Spanish recordings translated to English. The mean transcript length was 4244 ± 992 words. The overall accuracy was 98.2% ± 0.5%. On average, 46 words were missed in translation (1.09% error rate), 31 understanding-altering translation errors (0.73% error rate), and 0 catastrophic errors were made. There was no significant difference in English to Spanish or vice versa.

Conclusion: Voice to text translation devices using automatic speech recognition accurately translate recorded clinic conversations between Spanish and English with high accuracy and low incidence of errors impacting medical care or understanding. Further study should investigate additional languages, assess patient preferences and potential concerns with respect to device use, and compare these devices directly to medical interpreters in live clinic settings.

Keywords: Access to Care; Global neurosurgery; Pediatric Neurosurgery Clinic; Quality Improvement; Translation.