Hybridization of single-walled carbon nanotubes with plasmonic metamaterials leads to photonic media with an exceptionally strong ultrafast nonlinearity. This behavior is underpinned by strong coupling of the nanotube excitonic response to the weakly radiating Fano-type resonant plasmonic modes that can be tailored by metamaterial design.