Externally imposed loading has substantially different effects on the swelling of nematic elastomers in the high-temperature isotropic and low-temperature nematic states. In the isotropic state, the stretching drives a considerably large degree of further swelling, whereas the stretching-induced volume change in the nematic state is significantly suppressed. In the isotropic phase that favors the less anisotropic state, the further swelling occurs to reduce the shape anisotropy caused by the imposed elongation. In the nematic phase, no significant swelling is induced because further swelling decreases the nematic order enhanced by the applied stretching. These different loading effects in the isotropic and nematic states observed in the experiments are qualitatively described by a mean field theory.