Neurons in the medial superior olive (MSO) process microsecond interaural time differences, the major cue for localizing low-frequency sounds, by comparing the relative arrival time of binaural, glutamatergic excitatory inputs. This coincidence detection mechanism is additionally shaped by highly specialized glycinergic inhibition. Traditionally, it is assumed that the binaural inputs are conveyed by many independent fibers, but such an anatomical arrangement may decrease temporal precision. Short-term depression on the other hand might enhance temporal fidelity during ongoing activity. For the first time we show that binaural coincidence detection in MSO neurons may require surprisingly few but strong inputs, challenging long-held assumptions about mammalian coincidence detection. This study exclusively uses adult gerbils for in vitro electrophysiology, single-cell electroporation and immunohistochemistry to characterize the size and short-term plasticity of inputs to the MSO. We find that the excitatory and inhibitory inputs to the MSO are well balanced both in strength and short-term dynamics, redefining this fastest of all mammalian coincidence detector circuits.