A substantial proportion of raphe neurons are glutamatergic. However, little is known about how these glutamatergic neurons modulate the forebrain. We investigated how glutamatergic median raphe nucleus (MRN) input modulates the medial prefrontal cortex (mPFC), a critical component of fear circuitry. We show that vesicular glutamate transporter 3 (VGLUT3)-expressing MRN neurons activate VGLUT3- and somatostatin-expressing neurons in the mPFC. Consistent with this modulation of mPFC GABAergic neurons, activation of MRN (VGLUT3) neurons enhances GABAergic transmission in mPFC pyramidal neurons and attenuates fear memory in female but not male mice. Serotonin plays a key role in MRN (VGLUT3) neuron-mediated GABAergic plasticity in the mPFC. In agreement with these female-specific effects, we observed sex differences in glutamatergic transmission onto MRN (VGLUT3) neurons and in mPFC (VGLUT3) neuron-mediated dual release of glutamate and GABA. Our results demonstrate a cell type-specific modulation of the mPFC by MRN (VGLUT3) neurons and reveal a sex-specific role of this neuromodulation in mPFC synaptic plasticity.