In this paper, a general class of recurrent neural networks with time-varying delays is studied. Some novel and sufficient conditions are given to guarantee the global exponential stability of the equilibrium point and the existence of periodic solutions for such delayed neural networks. Comparing with some previous literature, in which the time-varying delays were assumed to be differentiable and their derivatives were simultaneously required to be not greater than 1, the restrictions on the time-varying delays are removed. Therefore, our results obtained here improve and extend some previously related results. Finally, two numerical examples are provided to illustrate our theorems.