We study a mathematical model of a single neuron with self-coupling. The model is based on the FitzHugh-Nagumo oscillator and an equation describing synaptic properties of the neuron. The analysis of the model is focused on its dynamics, depending on parameters characterizing synaptic time constants and external signals that affect the neuron. Applying Lyapunov exponents and bifurcation analysis, we point out the occurrence of parameter regions with different behavior such as bursting (chaotic or periodic), spiking, and multistable phenomena. Moreover, we can describe the dynamics of the model using an analytical approximation of the one-dimensional Poincaré map extracted from the numerical simulations.