In the mouse model of Fragile X syndrome, the Fmr1 knock-out, local excitation of layer 4 fast-spiking (FS) inhibitory neurons is robustly decreased by 50%, but the mechanisms mediating this change are unknown. Here, we performed recordings in acutely prepared slices obtained from Fmr1 "mosaic" mice, where Fmr1 is deleted in about half of all neurons, and we found that loss of presynaptic, but not postsynaptic, Fmr1 fully recapitulates the deficit. The change in connection strength is primarily due to a decrease in release probability indicating that FMRP normally positively regulates these processes. This change in presynaptic neurotransmitter release is observed both in the mosaic mice and in the constitutive Fmr1 knock-out mice. Manipulations in release probability enabled both the mimic and rescue of the impaired function in this synaptic pathway. Loss of presynaptic Fmr1 has no effect on excitatory synapses onto excitatory neurons, indicating a target cell-specific function for presynaptic FMRP. Finally, we demonstrate that the excitation decrement onto FS neurons also exists in layer 5 of the Fmr1 knock-out, suggesting a widespread role for presynaptic Fmr1 in the excitation of inhibitory neurons. In summary, we identify a novel function for presynaptic FMRP in promoting presynaptic neurotransmitter release, and we show that loss of this function accounts for impaired excitation of neocortical FS inhibitory neurons. These changes may contribute to the cognitive dysfunction and circuit hyperexcitability associated with Fragile X syndrome, including patients with complete deletion of FMRP and those with mosaic expression of FMRP.