This paper considers the problem of delay-dependent stability criteria for neural networks with time-varying delays. First, by constructing a newly augmented Lyapunov-Krasovskii functional, a less conservative stability criterion is established in terms of linear matrix inequalities. Second, by proposing novel activation function conditions which have not been proposed so far, further improved stability criteria are proposed. Finally, three numerical examples used in the literature are given to show the improvements over the existing criteria and the effectiveness of the proposed idea.