Interictal discharges (IIDs) are abnormal waveforms registered in the periods before or between seizures. IIDs that are initiated by GABAergic interneurons have not been mathematically modeled yet. In the present study, a mathematical model that describes the mechanisms of these discharges is proposed. The model is based on the experimental recordings of IIDs in pyramidal neurons of the rat entorhinal cortex and estimations of synaptic conductances during IIDs. IIDs were induced in cortico-hippocampal slices by applying an extracellular solution with 4-aminopyridine, high potassium, and low magnesium concentrations. Two different types of IIDs initiated by interneurons were observed. The first type of IID (IID1) was pure GABAergic. The second type of IID (IID2) was induced by GABAergic excitation and maintained by recurrent interactions of both GABA- and glutamatergic neuronal populations. The model employed the conductance-based refractory density (CBRD) approach, which accurately approximates the firing rate of a population of similar Hodgkin-Huxley-like neurons. The model of coupled excitatory and inhibitory populations includes AMPA, NMDA, and GABA-receptor-mediated synapses and gap junctions. These neurons receive both arbitrary deterministic input and individual colored Gaussian noise. Both types of IIDs were successfully reproduced in the model by setting two different depolarized levels for GABA-mediated current reversal potential. It was revealed that short-term synaptic depression is a crucial factor in ceasing each of the discharges, and it also determines their durations and frequencies.