Metallic nanostructures have the potential to modify the anti-Stokes emission of upconverting nanoparticles (UCNPs) by coupling their plasmon resonance with either the excitation or the emission wavelength of the UCNPs. In this regard gold nanoparticles (AuNPs) have often been used in sensors for UCNP luminescence quenching or enhancement, although systematic studies are still needed in order to design optimal UCNP-AuNP based biosensors. Amidst mixed experimental evidence of quenching or enhancement, two key factors arise: the nanoparticle distance and nanoparticle size. In this work, we synthesize AuNPs of different sizes to assess their influence on the luminescence of UCNPs. We find that strong luminescence quenching due to resonance energy transfer is preferentially achieved for small AuNPs, peaking at an optimal size. A further increase in the AuNP size is accompanied by a reduction of luminescence quenching due to an incipient plasmonic enhancement effect. This enhancement counterbalances the luminescence quenching effect at the biggest tested AuNP size. The experimental findings are theoretically validated by studying the decay rate of the UCNP emitters near a gold nanoparticle using both a classical phenomenological model and the finite-difference time-domain method. Results from this study establish general guidelines to consider when designing sensors based on UCNPs-AuNPs as donor-quencher pairs, and suggest the potential of plasmon-induced luminescence enhancement as a sensing strategy.