Aqueous microgels based on poly(N-vinylcaprolactam) with reversible temperature-induced volume transition are promising "smart" materials for various applications. In this work, the microgels are modified via acid-base interaction by wedge-shaped amphiphilic sulfonic acid molecules with alkyl chains of different lengths and an azobenzene group. In contrast to the pristine microgel the modified microgels retain colloidal stability in water and show different responses to the change of temperature and pH. The azobenzene group in the ligand molecules acts as a spectroscopic and kinetic probe sensing the microenvironment inside the microgel particles. Thus, the observed hyperchromicity upon heating suggests the enhancement of hydrophobicity with the increase of temperature. The hydrophobicity of the microgel interior increases with the increase of the modification degree as indicated by the increase of activation energy of the thermal Z/E isomerization of the azobenzene group.