We show that femtosecond laser pulse excitation of the orthoferrite ErFeO_{3} triggers pico- and subpicosecond dynamics of magnetic and electric dipoles associated with the low energy electronic states of the Er^{3+} ions. These dynamics are readily revealed by using polarization sensitive terahertz emission spectroscopy. It is shown that by changing the polarization of the femtosecond laser pulse one can excite either electric dipole-active or magnetic dipole-active transitions between the Kramers doublets of the ^{4}I_{15/2} ground state of the Er^{3+} (4f^{11}) ions. These observations serve as a proof of principle of polarization-selective control of both electric and magnetic degrees of freedom at terahertz frequencies, opening up new vistas for optical manipulation of magnetoelectric materials.