We propose a mechanism that couples matter fields to three-dimensional quantum gravity, which can be used for theories with a positive or negative cosmological constant. Our proposal is rooted in the Chern-Simons formulation of three-dimensional gravity and makes use of the Wilson spool, a collection of Wilson loops winding around closed paths of the background. We show that the Wilson spool correctly reproduces the one-loop determinant of a free massive scalar field on rotating black holes in AdS_{3} and Euclidean dS_{3} as G_{N}→0. Moreover, we describe how to incorporate quantum metric fluctuations into this formalism.