Dairy protein composition is known to influence the structure and the texture characteristics of yogurt. The objective of the present work was therefore to investigate the impact of protein composition, at a constant protein level, on the physicochemical properties of 4% fat flavored stirred yogurt and, more specifically, on the rheological properties, the microstructure, and the aroma release. The results showed that caseinate-enriched yogurt generally presented changes in their microstructure network and had a higher complex viscosity than whey protein-enriched yogurt. To a lesser extent, the release of the majority of aroma compounds was lower in caseinate-enriched yogurt. It was therefore possible to quantify physicochemical interactions between aroma compounds and proteins. The influence of gel structure on the flavor release was observed and was in agreement with sensory characteristics previously studied for these products.