This paper presents a simplified nonlinear model for Dynamic Synapse Neural Network (DSNN) which is based on nonlinear dynamics of neurons in the hippocampus, using a recurrent neural network. The proposed model will be utilized in place of DSNN for various applications which require simpler implementation and faster training, maintaining the same performance as a nonlinear system model, classifier, or pattern recognizer. This model was tested in two different structure and training methods, by learning the input-output relationship of a few DSNNs with sets of experimentally-determined coefficients. The results showed that this model can capture DSNN's complicated nonlinear dynamics in a temporal domain with less computational cost and faster training.