We report on the observation of electromagnetically induced transparency (EIT) and absorption (EIA) of highly excited Rydberg states in thermal Cs vapor using a four-step excitation scheme. The advantage of this four-step scheme is that the final transition to the Rydberg state has a large dipole moment and one can achieve similar Rabi frequencies to two- or three-step excitation schemes using two orders of magnitude less laser power. This scheme enables new applications such as dephasing free Rydberg excitation. The observed lineshapes are in good agreement with simulations based on multilevel optical Bloch equations.