The tradeoff between knowing when to seek greater rewards (exploration), and knowing when to settle (exploitation), is critical to success. One dispositional factor that may modulate this tradeoff is "grit." Gritty individuals tend to persist in the face of difficulty and consequently experience greater life success. It is possible that they may also experience a greater tendency to explore in a reward task. However, although most exploration/exploitation tasks manipulate beliefs about the presence/magnitude of rewards in the environment, the belief of one's ability to actually achieve a reward is also critical. As such, we investigated whether individuals higher in grit were more likely to explore, and how beliefs about the magnitude/presence of rewards, and the perceived ability to achieve a reward, modulated their exploration tendencies. Over two experiments, participants completed 4 different exploration/persistence tasks: two that tapped into participant beliefs about the presence/magnitude of rewards, and two that tapped into participant beliefs about their ability to achieve a reward. Participants also completed measures of dispositional grit (Experiment 1a and 1b), conscientiousness (Experiment 1b), and working memory (Experiment 1a and 1b). In both experiments, we found a relationship between the two "belief of rewards" tasks, as well as between the two "belief of ability" tasks, but performance was unrelated across the two types of task. We also found that dispositional grit was strongly associated with greater exploration, but only on the "belief of ability" tasks. Finally, in Experiment 1b we showed that conscientiousness better predicted exploration on the "belief of ability" tasks than grit, suggesting that it is not grittiness per se that is associated with exploration. Overall, our findings showed that individuals high in grit/conscientiousness are more likely to explore, but only when there is a known reward available that they believe they have the ability to achieve.