Theoretical and numerical models of active Janus particles commonly assume that the metallo-dielectric interface is parallel to the driving applied electric field. However, our experimental observations indicate that the equilibrium angle of orientation of electrokinetically driven Janus particles varies as a function of the frequency and voltage of the applied electric field. Here, we quantify the variation of the orientation with respect to the electric field and demonstrate that the equilibrium position represents the interplay between gravitational, electrostatic and electrohydrodynamic torques. The latter two categories are functions of the applied field (frequency, voltage) as well as the height of the particle above the substrate. Maximum departure from the alignment with the electric field occurs at low frequencies characteristic of induced-charge electrophoresis and at low voltages where gravity dominates the electrostatic and electrohydrodynamic torques. The departure of the interface from alignment with the electric field is shown to decrease particle mobility through comparison of freely suspended Janus particles subject only to electrical forcing and magnetized Janus particles in which magnetic torque is used to align the interface with the electric field. Consideration of the role of gravitational torque and particle-wall interactions could account for some discrepancies between theory, numerics and experiment in active matter systems.