Due to the great advances in mobility techniques, an increasing number of point-of-interest (POI)-related services have emerged, which could help users to navigate or predict POIs that may be interesting. Obviously, predicting POIs is a challenging task, mainly because of the complicated sequential transition regularities, and the heterogeneity and sparsity of the collected trajectory data. Most prior studies on successive POI recommendation mainly focused on modeling the correlation among POIs based on users' check-in data. However, given a user's check-in sequence, generally, the relationship between two consecutive POIs is usually both time and distance subtle. In this article, we propose a novel POI recommendation system to capture and learn the complicated sequential transitions by incorporating time and distance irregularity. In addition, we propose a feasible way to dynamically weight the decay values into the model learning process. The learned awareness weights offer an easy-to-interpret way to translate how much each context is emphasized in the prediction process. The performance evaluations are conducted on real mobility datasets to demonstrate the effectiveness and practicability of the POI recommendations. The experimental results show that the proposed methods significantly outperform the state-of-the-art models in all metrics.