The relation between the orientation of the magnetic field and the flopped ferroelectric polarization has been investigated for multiferroic MnWO4. The ferroelectric single-domain state is retained across the polarization flop process when the direction of the applied magnetic field slightly deviates from the b axis within the ab plane. Furthermore, the electric polarization in the high-field P parallela phase is reversed when the P parallelb-to-P parallela transition takes place while decreasing and increasing the magnetic fields oppositely canted from the b axis. These results indicate that the symmetry breaking induced by a canted magnetic field determines the direction of the polarization flop, which corresponds to the direction of the vector spin chirality. The stability of the magnetoelectric domain walls in a canted magnetic field play a key role in the directional control of the electric polarization flop phenomenon.