Acrylamide-based, thermoresponsive core-shell microgels with a linear phase transition region are used as improved carriers for catalytically active silver nanoparticles in the present study. In this context, we investigated the swelling behavior of the carriers and the stability of the silver nanoparticles inside the polymer network with photon correlation spectroscopy, transmission electron microscopy, and by following the surface plasmon resonance of the nanoparticles. Depending on the cross-linker content of the microgel core, we observed very good stability of the nanoparticles inside the microgel network, with nearly no bleeding or aggregation of the nanoparticles over several weeks for core cross-linker contents of 5 and 10 mol %. The architecture of the hybrid particles in the swollen state was investigated with cryogenic transmission electron microscopy. The particles exhibit a core-shell structure, with the silver nanoparticles located mainly at the interface between the core and shell. This architecture was not used before and seems to grant advanced stability to the nanoparticles inside the network in combination with good switchability of the catalytic activity. This was measured by following the reduction of 4-nitrophenole, which is a well-studied model reaction. The obtained Arrhenius plots show that similar to previous works, the swelling of the core and shell can influence the catalytic activity of the silver nanoparticles. As mentioned before, the cross-linker content of the core seems to be a very important parameter for the switchability of the catalytic activity. A higher cross-linker content of the core seems to be connected to a stronger influence of the carrier swelling degree on the catalytic activity of the silver nanoparticles.