We compared experimental and simulated data to investigate the phase separation and water channel formation of proton exchange membranes (PEMs) for fuel cell. Sulfonated block copolyimides (SPIs) were adopted as model polymers for experiments and simulations, and Nafion was used as a reference. Nafion and SPIs were observed to have different microscopic structures such as constituent atoms, backbone rigidity, and the locations of sulfonic acid groups, all of which significantly affect phenomenological properties at the macroscopic level such as density, water uptake, and proton conductivity. In particular, SPIs show much weaker microphase separation than Nafion, mainly due to the lower mobility of sulfonic acid groups and the existence of acceptable sites for hydrogen bonding even in hydrophobic segments, which impedes water channel formation for proton transport. As a result, the phase separation behavior and the resulting water channel formation are the major factors affecting macroscopic properties of PEMs such as water uptake and proton transport.