How experts and novices judge other people's knowledgeability from language use

Alexander H Bower; Nicole Han; Ansh Soni; Miguel P Eckstein; Mark Steyvers

doi:10.3758/s13423-023-02433-9

How experts and novices judge other people's knowledgeability from language use

Psychon Bull Rev. 2024 Jan 4. doi: 10.3758/s13423-023-02433-9. Online ahead of print.

Authors

Alexander H Bower¹, Nicole Han^{2

3}, Ansh Soni², Miguel P Eckstein^{2

4

5

3}, Mark Steyvers⁶

Affiliations

¹ Department of Cognitive Sciences, University of California, Irvine, CA, USA.
² Department of Psychological and Brain Sciences, University of California, Santa Barbara, CA, USA.
³ Institute for Collaborative Biotechnologies, University of California, Santa Barbara, CA, USA.
⁴ Department of Electrical and Computer Engineering, University of California, Santa Barbara, CA, USA.
⁵ Department of Computer Science, University of California, Santa Barbara, CA, USA.
⁶ Department of Cognitive Sciences, University of California, Irvine, CA, USA. mark.steyvers@uci.edu.

PMID: 38177890
DOI: 10.3758/s13423-023-02433-9

Abstract

How accurate are people in judging someone else's knowledge based on their language use, and do more knowledgeable people use different cues to make these judgments? We address this by recruiting a group of participants ("informants") to answer general knowledge questions and describe various images belonging to different categories (e.g., cartoons, basketball). A second group of participants ("evaluators") also answer general knowledge questions and decide who is more knowledgeable within pairs of informants, based on these descriptions. Evaluators perform above chance at identifying the most knowledgeable informants (65% with only one description available). The less knowledgeable evaluators base their decisions on the number of specific statements, regardless of whether the statements are true or false. The more knowledgeable evaluators treat true and false statements differently and penalize the knowledge they attribute to informants who produce specific yet false statements. Our findings demonstrate the power of a few words when assessing others' knowledge and have implications for how misinformation is processed differently between experts and novices.

Keywords: Decision-making; Knowledge; Language; Mindreading; Theory of mind.