Speech training paradigms aim to maximise learning outcomes by manipulating external factors such as talker variability. However, not all individuals may benefit from such manipulations because subject-external factors interact with subject-internal ones (e.g., aptitude) to determine speech perception and/or learning success. In a previous tone learning study, high-aptitude individuals benefitted from talker variability, whereas low-aptitude individuals were impaired. Because increases in cognitive load have been shown to hinder speech perception in mixed-talker conditions, it has been proposed that resolving talker variability requires cognitive resources. This proposal leads to the hypothesis that low-aptitude individuals do not use their cognitive resources as efficiently as those with high aptitude. Here, high- and low-aptitude subjects identified pitch contours spoken by multiple talkers under high and low cognitive load conditions established by a secondary task. While high-aptitude listeners outperformed low-aptitude listeners across load conditions, only low-aptitude listeners were impaired by increased cognitive load. The findings suggest that low-aptitude listeners either have fewer available cognitive resources or are poorer at allocating attention to the signal. Therefore, cognitive load is an important factor when considering individual differences in speech perception and training paradigms.