New species may arise via hybridization and without a change in ploidy. This process, termed homoploid hybrid speciation, is theoretically difficult because it requires the development of reproductive barriers in sympatry or parapatry. Theory suggests that isolation may arise through rapid karyotypic evolution and/or ecological divergence of hybrid neospecies. Here, we investigate the role of karyotypic change in homoploid hybrid speciation by generating detailed genetic linkage maps for three hybrid sunflower species, Helianthus anomalus, H. deserticola, and H. paradoxus, and comparing these maps to those previously generated for the parental species, H. annuus and H. petiolaris. We also conduct a quantitative trait locus (QTL) analysis of pollen fertility in a BC2 population between the parental species and assess levels of pollen and seed fertility in all cross-combinations of the hybrid and parental species. The three hybrid species are massively divergent from their parental species in karyotype; gene order differences were observed for between 9 and 11 linkage groups (of 17 total), depending on the comparison. About one-third of the karyoypic differences arose through the sorting of chromosomal rearrangements that differentiate the parental species, but the remainder appear to have arisen de novo (six breakages/six fusions in H. anomalus, four breakages/three fusions in H. deserticola, and five breakages/five fusions in H. paradoxus). QTL analyses indicate that the karyotypic differences contribute to reproductive isolation. Nine of 11 pollen viability QTL occur on rearranged chromosomes and all but one map close to a rearrangement breakpoint. Finally, pollen and seed fertility estimates for F1's between the hybrid and parental species fall below 11%, which is sufficient for evolutionary independence of the hybrid neospecies.