Transformable double hydrophilic block copolymer assemblies are valid as a biocompatible smart macromolecular system. The molecular mechanisms in the spontaneous assembly of double zwitterionic diblock copolymers composed of a poly(carboxybetaine methacrylate) (PCB2) and a poly(sulfobetaine methacrylate) (PSB4) chains (PCB2-b-PSB4) were investigated by the modulation of the aggregates in response to nondetergent zwitterions. The PCB2-b-PSB4 diblock copolymers with a high degree of polymerization PSB4 block produced aggregates in salt-free water through "zwitterion-specific" interactions. The PCB2-b-PSB4 aggregates were dissociated by the addition of nondetergent sulfobetaine (SB4) and carboxybetaine (CB2) molecules, while the aggregates showed different aggregation modulation processes for SB4 and CB2. Zwitterions with different charged groups from SB4 and CB2, glycine and taurine, hardly disrupted the PCB2-b-PSB4 aggregates. The PCB2-b-PSB4 aggregate modulation efficiency of SBs associated with the intercharge hydrocarbon spacer length (CSL) rather than the symmetry with the SB in the PSB chain. These zwitterion-specific modulation behaviors were rationalized based on the nature of zwitterions including partial charge density, dipole moment, and hydrophobic interactions depending on the charged groups and CSL.