To develop a stable and marker-free Lactobacillus strain useful for the expression of vaccines, we developed a temperature-sensitive suicide plasmid with expression cassettes containing an HCE promoter, a PgsA anchor, the alpha-toxin gene, and an rrnB T1T2 terminator (PPαT) that uses a 5-fluorouracil (5-FU) counterselectable marker for Lactobacillus casei. Three strains containing the correct PPαT expression cassettes were produced via the selective pressure of 5-FU screening. We confirmed that the upp gene was deleted and that the PPαT expression cassettes were inserted into the upp site of L. casei ATCC 393 by genomic PCR amplification and sequencing. 5-FU resistance in recombinant bacteria could be stably inherited for as long as 40 generations following insertion. However, bacteria containing the integrated DNA grew more slowly than wild-type L. casei. An indirect enzyme-linked immunosorbent assay (ELISA) analysis demonstrated that the alpha-toxin gene was expressed. Also, we visualized expression of the protein on the surface of L. casei cells using laser confocal microscopy. These results taken together demonstrate that these recombinant bacteria should provide a safe tool for effective vaccine production.