Seedlessness is one of the important traits in citrus breeding. Male sterility derived from Satsuma mandarin (Citrus unshiu) has been used in Japanese citrus breeding programs to obtain seedless cultivars. The efficiency of seedless cultivar breeding would be improved by developing a selection marker linked to seedlessness. In this study, we performed QTL mapping in 'Okitsu No. 46' × 'Okitsu No. 56' (O46-O56) crosses for the number of pollen grains per anther (NPG) and apparent pollen fertility (APF), two traits used as an index of male sterility, and detected a candidate QTL for NPG (MS-P1) on linkage group 8 with a significant LOD score (7.31) and 47% of variance explained. The QTL for APF (MS-F1) was detected on linkage group 6 with a significant LOD score (5.71) and 63.6% of variance explained. The role of both MS-P1 in reducing NPG and MS-F1 in decreasing APF were confirmed with the 'Okitsu No.46' × 'Kara' (O46-K) cross. Pedigree analysis inferred that both MS-P1 and MS-F1 in 'Okitsu No. 46' were derived from kunenbo (Citrus nobilis) through hassaku (C. hassaku) and 'Sweet Spring'. Cytoplasm analysis revealed that both male-sterile 'Sweet Spring' and 'Okitsu No. 46' have cytoplasm derived from Kishu (C. kinokuni hort. ex Tanaka), but the cytoplasm of male-sterile kunenbo and hassaku were derived from other varieties rather than Kishu. These results suggest that MS-P1 and MS-F1 primarily reduce the NPG and decrease APF, but their expression requires a cytoplasm derived from Kishu. These findings will improve our understanding of the molecular mechanism of male sterility in citrus and help to develop a DNA marker for seedless breeding in citrus.