The stickiness effect is a fundamental feature of quasi-integrable Hamiltonian systems. We propose the use of an entropy-based measure of the recurrence plots (RPs), namely, the entropy of the distribution of the recurrence times (estimated from the RP), to characterize the dynamics of a typical quasi-integrable Hamiltonian system with coexisting regular and chaotic regions. We show that the recurrence time entropy (RTE) is positively correlated to the largest Lyapunov exponent, with a high correlation coefficient. We obtain a multi-modal distribution of the finite-time RTE and find that each mode corresponds to the motion around islands of different hierarchical levels.