Visuospatial working memory (VSWM) helps track the identity and location of people during social interactions. Previous work showed better VSWM when all faces at encoding displayed a happy compared to an angry expression, reflecting a prosocial preference for monitoring who was where. However, social environments are not typically uniform, and certain expressions may more strongly compete for and bias face monitoring according to valence and/or arousal properties. Here, we used heterogeneous encoding displays in which two faces shared one emotion and two shared another, and asked participants to relocate a central neutral probe face after a blank delay. When considering the emotion of the probed face independently of the co-occurring emotion at encoding, an overall happy benefit was replicated. However, accuracy was modulated by the nonprobed emotion, with a relocation benefit for angry over sad, happy over fearful, and sad over happy faces. These effects did not depend on encoding fixation time, stimulus arousal, perceptual similarity, or response bias. Thus, emotional competition for faces in VSWM is complex and appears to rely on more than simple arousal- or valence-biased mechanisms. We propose a "social value (SV)" account to better explain when and why certain emotions may be prioritized in VSWM. (PsycInfo Database Record (c) 2024 APA, all rights reserved).