How persuasive is AI-generated propaganda?

Josh A Goldstein; Jason Chao; Shelby Grossman; Alex Stamos; Michael Tomz

doi:10.1093/pnasnexus/pgae034

How persuasive is AI-generated propaganda?

PNAS Nexus. 2024 Feb 20;3(2):pgae034. doi: 10.1093/pnasnexus/pgae034. eCollection 2024 Feb.

Authors

Josh A Goldstein¹, Jason Chao², Shelby Grossman², Alex Stamos², Michael Tomz³

Affiliations

¹ Center for Security and Emerging Technology, Georgetown University, Washington, DC 20001, USA.
² Stanford Internet Observatory, Stanford University, Stanford, CA 94305, USA.
³ Department of Political Science and Stanford Institute for Economic Policy Research, Stanford University, Stanford, CA 94305, USA.

Abstract

Can large language models, a form of artificial intelligence (AI), generate persuasive propaganda? We conducted a preregistered survey experiment of US respondents to investigate the persuasiveness of news articles written by foreign propagandists compared to content generated by GPT-3 davinci (a large language model). We found that GPT-3 can create highly persuasive text as measured by participants' agreement with propaganda theses. We further investigated whether a person fluent in English could improve propaganda persuasiveness. Editing the prompt fed to GPT-3 and/or curating GPT-3's output made GPT-3 even more persuasive, and, under certain conditions, as persuasive as the original propaganda. Our findings suggest that propagandists could use AI to create convincing content with limited effort.

Keywords: artificial intelligence; language models; propaganda; survey experiment.