The tapetum plays an important role in anther development by providing necessary enzymes and nutrients for pollen development. However, it is difficult to identify tapetum-specific genes on a large-scale because of the difficulty of separating tapetum cells from other anther tissues. Here, we reported the identification of tapetum-specific genes by comparing the gene expression patterns of four male sterile (MS) lines of Brassica oleracea. The abortive phenotypes of the four MS lines revealed different defects in tapetum and pollen development but normal anther wall development when observed by transmission electron microscopy. These tapetum displayed continuous defective characteristics throughout the anther developmental stages. The transcriptome from flower buds, covering all anther developmental stages, was analyzed and bioinformatics analyses exploring tapetum development-related genes were performed. We identified 1,005 genes differentially expressed in at least one of the MS lines and 104 were non-pollen expressed genes (NPGs). Most of the identified NPGs were tapetum-specific genes considering that anther walls were normally developed in all four MS lines. Among the 104 NPGs, 22 genes were previously reported as being involved in tapetum development. We further separated the expressed NPGs into different developmental stages based on the MS defects. The data obtained in this study are not only informative for research on tapetum development in B. oleracea, but are also useful for genetic pathway research in other related species.