When a pure droplet evaporates inside an elastic medium, two instabilities are typically observed. As the droplet shrinks, the elastic medium needs to deform and elastic tension builds up. At a critical strain, accumulated tension at the gel-droplet interface is released by developing creases. The droplet keeps shrinking beyond this point, pulling the elastic network and therefore decreasing the pressure in the liquid phase. This drives the liquid phase into a metastable state, and leads to the second instability: the nucleation of a vapour bubble in the liquid phase by cavitation. These instabilities consistently occur in the described order whenever a pure liquid (water, in this case) is used. The presence of colloidal particles inside droplets is common both in vitro and in natural environments, and they can change such phenomenology significantly by stimulating cavitation events before any creasing instability. In this work, we study the role of colloidal particle size and concentration on the early inception of cavitation in water droplets in an elastic medium. Our results reveal an unexpected dependence with the particle size and with the size distribution of the colloidal particles. Given the simplicity and reliability of the system and preparation, the method described here could be eventually used to measure tensile strengths of particle solutions with accuracy.