Controlling magnetism with electric fields is a key challenge to develop future energy-efficient devices. The present magnetic information technology is mainly based on writing processes requiring either local magnetic fields or spin torques, but it has also been demonstrated that magnetic properties can be altered on the application of electric fields. This has been ascribed to changes in magnetocrystalline anisotropy caused by spin-dependent screening and modifications of the band structure, changes in atom positions or differences in hybridization with an adjacent oxide layer. However, the switching between states related by time reversal, for example magnetization up and down as used in the present technology, is not straightforward because the electric field does not break time-reversal symmetry. Several workarounds have been applied to toggle between bistable magnetic states with electric fields, including changes of material composition as a result of electric fields. Here we demonstrate that local electric fields can be used to switch reversibly between a magnetic skyrmion and the ferromagnetic state. These two states are topologically inequivalent, and we find that the direction of the electric field directly determines the final state. This observation establishes the possibility to combine electric-field writing with the recently envisaged skyrmion racetrack-type memories.