The substantial gender gap in the science, technology, engineering, and mathematics (STEM) workforce can be traced back to the underrepresentation of women at various milestones in the career pathway. Calculus is a necessary step in this pathway and has been shown to often dissuade people from pursuing STEM fields. We examine the characteristics of students who begin college interested in STEM and either persist or switch out of the calculus sequence after taking Calculus I, and hence either continue to pursue a STEM major or are dissuaded from STEM disciplines. The data come from a unique, national survey focused on mainstream college calculus. Our analyses show that, while controlling for academic preparedness, career intentions, and instruction, the odds of a woman being dissuaded from continuing in calculus is 1.5 times greater than that for a man. Furthermore, women report they do not understand the course material well enough to continue significantly more often than men. When comparing women and men with above-average mathematical abilities and preparedness, we find women start and end the term with significantly lower mathematical confidence than men. This suggests a lack of mathematical confidence, rather than a lack of mathematically ability, may be responsible for the high departure rate of women. While it would be ideal to increase interest and participation of women in STEM at all stages of their careers, our findings indicate that if women persisted in STEM at the same rate as men starting in Calculus I, the number of women entering the STEM workforce would increase by 75%.