The synthesis of composite microgels consisting of pH-responsive latexes with gold nanoparticles was investigated along with the optical properties of the products. The gold nanoparticles were deposited by wet chemical reduction from gold ions adsorbed in cross-linked poly(2-vinylpyridine) latexes, by which the mean particle size of the gold nanoparticles could be systematically controlled over a range of 10-30 nm simply by varying the reduction rate. Microscopic analysis showed that the gold nanoparticles were formed only on the surface of the microgels, resulting from diffusion of the gold ions from the interior to the surface of the microgels during reduction treatment. The resulting nanocomposites preserved the pH-responsive properties of the pure latexes. The degree of plasmon coupling, originating from dipole interactions among the gold nanoparticles, was dependent on the size of the nanoparticles and could be reversibly controlled by varying the pH of the aqueous solution. The process allowed independent control of the size and interparticle distance among gold nanoparticles, an ability that is important in increasing the fundamental understanding of the structure-dependent properties of gold nanoparticles and also for biological applications using functionalized composite latexes/microgels.