The origin of the sum-frequency generation (SFG) signal of the water bending mode has been controversially debated in the past decade. Unveiling the origin of the signal is essential, because different assignments lead to different views on the molecular structure of interfacial water. Here, we combine collinear heterodyne-detected SFG spectroscopy at the water-charged lipid interfaces with systematic variation of the salt concentration. The results show that the bending mode response is of a dipolar, rather than a quadrupolar, nature and allows us to disentangle the response of water in the Stern and the diffuse layers. While the diffuse layer response is identical for the oppositely charged surfaces, the Stern layer responses reflect interfacial hydrogen bonding. Our findings thus corroborate that the water bending mode signal is a suitable probe for the structure of interfacial water.