This work presents a hybrid auxiliary density functional theory (ADFT) study of the neutral and hexaanionic C104 and C106 fullerenes with the aim to determine their ground state structures. To this end, all C104 and C106 fullerene structures that obey the isolated pentagon rule (IPR) were optimized with the Perdew-Burke-Ernzerhof generalized gradient approximation followed by a single-point energy calculation with the PBE0 hybrid functional. Our studies show that this composite approach yields relative energies of giant fullerenes that are accurate to around 1 kcal/mol. As a result, the ground states of C104, C1046-, and C1066- can be assigned to the isomers 234:Cs, 821:D2, and 891:Cs, respectively. On the other hand, the energetically lowest lying IPR isomers of C106, 331:Cs, 1194:C2, 534:C1 are separated by less than 1 kcal/mol which makes an unequivocal ground state assignment by hybrid DFT methods impossible. To guide future experiments, we also report the simulated IR and Raman spectra of the most stable neutral and hexaanionic C104 and C106 fullerenes.