Isolates of Aspergillus species are able to produce a large number of secondary metabolites. The profiles of biosynthetic families of secondary metabolites are species specific, whereas individual secondary metabolite families can occur in other species, even those phylogenetically and ecologically unrelated to Aspergillus. Furthermore, there is a high degree of chemo-consistency from isolate to isolate in a species even though certain metabolite gene clusters are silenced in some isolates. Genome sequencing projects have shown that the diversity of secondary metabolites is much larger in each species than previously thought. The potential of finding even further new bioactive drug candidates in Aspergillus is evident, despite the fact that many secondary metabolites have already been structure elucidated and chemotaxonomic studies have shown that many new secondary metabolites have yet to be characterized. The genus Aspergillus is cladistically holophyletic but phenotypically polythetic and very diverse and is associated to quite different sexual states. Following the one fungus one name system, the genus Aspergillus is restricted to a holophyletic clade that include the morphologically different genera Aspergillus, Dichotomomyces, Phialosimplex, Polypaecilum and Cristaspora. Secondary metabolites common between the subgenera and sections of Aspergillus are surprisingly few, but many metabolites are common to a majority of species within the sections. We call small molecule extrolites in the same biosynthetic family isoextrolites. However, it appears that secondary metabolites from one Aspergillus section have analogous metabolites in other sections (here also called heteroisoextrolites). In this review, we give a genus-wide overview of secondary metabolite production in Aspergillus species. Extrolites appear to have evolved because of ecological challenges rather than being inherited from ancestral species, at least when comparing the species in the different sections of Aspergillus. Within the Aspergillus sections, secondary metabolite pathways seem to inherit from ancestral species, but the profiles of these secondary metabolites are shaped by the biotic and abiotic environment. We hypothesize that many new and unique section-specific small molecule extrolites in each of the Aspergillus will be discovered.