The dynamics of an active walker in a harmonic potential is studied experimentally, numerically, and theoretically. At odds with usual models of self-propelled particles, we identify two dynamical states for which the particle condensates at a finite distance from the trap center. In the first state, also found in other systems, the particle points radially outward from the trap, while diffusing along the azimuthal direction. In the second state, the particle performs circular orbits around the center of the trap. We show that self-alignment, taking the form of a torque coupling the particle orientation and velocity, is responsible for the emergence of this second dynamical state. The transition between the two states is controlled by the persistence of the particle orientation. At low inertia, the transition is continuous. For large inertia, the transition is discontinuous and a coexistence regime with intermittent dynamics develops. The two states survive in the overdamped limit or when the particle is confined by a curved hard wall.