We explore how a magnetic field breaks the symmetry of an initially spherical giant liposome filled with a magnetic colloid. The condition of rotational symmetry along the field axis leads either to a prolate or to an oblate ellipsoid. We demonstrate that an electrostatic interaction between the nanoparticles and the membrane triggers the shape transition.