In this study, we proposed an online learning method using spike-timing dependent plasticity (STDP) whose operation is analogous to gradient descent, the most successful learning algorithm for nonspiking artificial neural networks (ANNs). With a model of a 4-terminal synaptic transistor we previously reported, a single-layer neural network implemented on the cross-point array was simulated by MATLAB to train binary MNIST samples with gradient descent algorithm. In addition, a proposed pulse scheme based on STDP was used to train the same network by applying teaching pulses having positive and negative timing differences with respect to input pulses to the back gate of the synaptic transistors. By comparing the extracted synaptic weight maps from both methods, therefore, the network trained by gradient descent was almost equally reproduced by the proposed method which was performed fully on hardware without computer calculation.