Natural competence allows bacteria to respond to environmental and nutritional cues by taking up free DNA from their surroundings, thus gaining both nutrients and genetic information. In the Gram-negative bacterium Haemophilus influenzae, the genes needed for DNA uptake are induced by the CRP and Sxy transcription factors in response to lack of preferred carbon sources and nucleotide precursors. Here we show that one of these genes, HI0659, encodes the antitoxin of a competence-regulated toxin-antitoxin operon ('toxTA'), likely acquired by horizontal gene transfer from a Streptococcus species. Deletion of the putative toxin (HI0660) restores uptake to the antitoxin mutant. The full toxTA operon was present in only 17 of the 181 strains we examined; complete deletion was seen in 22 strains and deletions removing parts of the toxin gene in 142 others. In addition to the expected Sxy- and CRP-dependent-competence promoter, HI0659/660 transcript analysis using RNA-seq identified an internal antitoxin-repressed promoter whose transcription starts within toxT and will yield nonfunctional protein. We propose that the most likely effect of unopposed toxin expression is non-specific cleavage of mRNAs and arrest or death of competent cells in the culture. Although the high frequency of toxT and toxTA deletions suggests that this competence-regulated toxin-antitoxin system may be mildly deleterious, it could also facilitate downregulation of protein synthesis and recycling of nucleotides under starvation conditions. Although our analyses were focused on the effects of toxTA, the RNA-seq dataset will be a useful resource for further investigations into competence regulation.