Interneurons are critical for the proper functioning of neural circuits. While often morphologically complex, their dendrites have been ignored for decades, treating them as linear point neurons. Exciting new findings reveal complex, non-linear dendritic computations that call for a new theory of interneuron arithmetic. Using detailed biophysical models, we predict that dendrites of FS basket cells in both hippocampus and prefrontal cortex come in two flavors: supralinear, supporting local sodium spikes within large-volume branches and sublinear, in small-volume branches. Synaptic activation of varying sets of these dendrites leads to somatic firing variability that cannot be fully explained by the point neuron reduction. Instead, a 2-stage artificial neural network (ANN), with sub- and supralinear hidden nodes, captures most of the variance. Reduced neuronal circuit modeling suggest that this bi-modal, 2-stage integration in FS basket cells confers substantial resource savings in memory encoding as well as the linking of memories across time.