Using Machine Translation and Post-Editing in the TRAPD Approach: Effects on the Quality of Translated Survey Texts

Diana Zavala-Rojas; Dorothée Behr; Brita Dorer; Danielly Sorato; Veronika Keck

doi:10.1093/poq/nfad060

Using Machine Translation and Post-Editing in the TRAPD Approach: Effects on the Quality of Translated Survey Texts

Public Opin Q. 2024 Mar 28;88(1):123-148. doi: 10.1093/poq/nfad060. eCollection 2024 Spring.

Authors

Diana Zavala-Rojas¹, Dorothée Behr², Brita Dorer³, Danielly Sorato⁴, Veronika Keck⁵

Affiliations

¹ Principal Investigator of the European Social Survey ERIC, Universitat Pompeu Fabra, Barcelona, Spain; and Deputy Director of the Research and Expertise Centre, Survey Methodology in the Political and Social Sciences Department, Universitat Pompeu Fabra, Barcelona, Spain.
² Head of Team, Cross-Cultural Survey Methods, Survey Design and Methodology Department, GESIS-Leibniz Institute for the Social Sciences, Mannheim, Germany.
³ Head of the Translation Workpackage, European Social Survey ERIC, Mannheim, Germany; and Senior Researcher, Survey Design and Methodology Department, GESIS-Leibniz Institute for the Social Sciences, Mannheim, Germany.
⁴ Researcher, Research and Expertise Centre for Survey Methodology, Political and Social Sciences Department, Universitat Pompeu Fabra, Barcelona, Spain; and PhD Candidate, Department of Translation and Language Sciences, Universitat Pompeu Fabra, Barcelona, Spain.
⁵ Senior Client Training Consultant, The Nielsen Company (Germany) GmbH (NielsenIQ), Frankfurt am Main, Germany.

Abstract

A highly controlled experimental setting using a sample of questions from the European Social Survey (ESS) and European Values Study (EVS) was used to test the effects of integrating machine translation and post-editing into the Translation, Review, Adjudication, Pretesting, and Documentation (TRAPD) approach in survey translation. Four experiments were conducted in total, two concerning the language pair English-German and two in the language pair English-Russian. The overall results of this study are positive for integrating machine translation and post-editing into the TRAPD process, when translating survey questionnaires. The experiments show evidence that in German and Russian languages and for a sample of ESS and EVS survey questions, the effect of integrating machine translation and post-editing on the quality of the review outputs-with quality understood as texts output with the fewest errors possible-can hardly be distinguished from the quality that derives from the setting with human translations only.