The Spot the Troll Quiz game increases accuracy in discerning between real and inauthentic social media accounts

Jeffrey Lees; John A Banas; Darren Linvill; Patrick C Meirick; Patrick Warren

doi:10.1093/pnasnexus/pgad094

The Spot the Troll Quiz game increases accuracy in discerning between real and inauthentic social media accounts

PNAS Nexus. 2023 Mar 22;2(4):pgad094. doi: 10.1093/pnasnexus/pgad094. eCollection 2023 Apr.

Authors

Jeffrey Lees^{1

2

3}, John A Banas⁴, Darren Linvill^{2

5}, Patrick C Meirick⁴, Patrick Warren^{1

2}

Affiliations

¹ John E. Walker Department of Economics, Clemson University, Clemson, SC 29634, USA.
² Media Forensics Hub, Clemson University, Clemson, SC 29634, USA.
³ Andlinger Center for Energy and the Environment, Princeton University, Princeton, NJ 08540, USA.
⁴ Department of Communication, University of Oklahoma, Norman, OK 73019, USA.
⁵ Department of Communication, Clemson University, Clemson, SC 29634, USA.

Abstract

The proliferation of political mis/disinformation on social media has led many scholars to embrace "inoculation" techniques, where individuals are trained to identify the signs of low-veracity information prior to exposure. Coordinated information operations frequently spread mis/disinformation through inauthentic or "troll" accounts that appear to be trustworthy members to the targeted polity, as in Russia's attempts to influence the 2016 US presidential election. We experimentally tested the efficacy of inoculation against inauthentic online actors, using the Spot the Troll Quiz, a free, online educational tool that teaches how to spot markers of inauthenticity. Inoculation works in this setting. Across an online US nationally representative sample (N = 2,847), which also oversampled older adults, we find that taking the Spot the Troll Quiz (vs. playing a simple game) significantly increases participants' accuracy in identifying trolls among a set of Twitter accounts that are novel to participants. This inoculation also reduces participants' self-efficacy in identifying inauthentic accounts and reduced the perceived reliability of fake news headlines, although it had no effect on affective polarization. And while accuracy in the novel troll-spotting task is negatively associated with age and Republican party identification, the Quiz is equally effective on older adults and Republicans as it was on younger adults and Democrats. In the field, a convenience set of Twitter users who posted their Spot the Troll Quiz results in the fall of 2020 (N = 505) reduced their rate of retweeting in the period after the Quiz, with no impact on original tweeting.

Keywords: Twitter; disinformation; inoculation; misinformation; social media; trolls.