The Rashba effect is fundamental to the physics of two-dimensional electron systems and underlies a variety of spintronic phenomena. It has been proposed that the formation of Rashba-type spin splittings originates microscopically from the existence of orbital angular momentum (OAM) in the Bloch wave functions. Here, we present detailed experimental evidence for this OAM-based origin of the Rashba effect by angle-resolved photoemission (ARPES) and two-photon photoemission experiments for a monolayer AgTe on Ag(111). Using quantitative low-energy electron diffraction analysis, we determine the structural parameters and the stacking of the honeycomb overlayer with picometer precision. Based on an orbital-symmetry analysis in ARPES and supported by first-principles calculations, we unequivocally relate the presence and absence of Rashba-type spin splittings in different bands of AgTe to the existence of OAM.