In the task of pedestrian trajectory prediction, social interaction could be one of the most complicated factors since it is difficult to be interpreted through simple rules. Recent studies have shown a great ability of LSTM networks in learning social behaviors from datasets, e.g., introducing LSTM hidden states of the neighbors at the last time step into LSTM recursion. However, those methods depend on previous neighboring features which lead to a delayed observation. In this paper, we propose a data-driven states refinement LSTM network (SR-LSTM) to enable the utilization of the current intention of neighbors through a message passing framework. Moreover, the model performs in the form of self-updating by jointly refining the current states of all participants, rather than an input-output mechanism served by feature concatenation. In the process of states refinement, a social-aware information selection module consisting of an element-wise motion gate and a pedestrian-wise attention is designed to serve as the guidance of the message passing process. Considering the pedestrian walking space as a graph where each pedestrian is a node and each pedestrian pair with an edge, spatial-edge LSTMs are further exploited to enhance the model capacity, where two kinds of LSTMs interact with each other so that states of them are interactively refined. Experimental results on four widely used pedestrian trajectory datasets, ETH, UCY, PWPD, and NYGC demonstrate the effectiveness of the proposed model.