The excitonic fine structure splitting describes the splitting of the bright excitons as a consequence of the atomistic symmetry of the lattice and the electron-hole exchange interaction. Efforts are underway to eliminate this natural splitting by external constraints in order to use quantum dots in quantum optics. We show by million atom empirical pseudopotential calculations that for realistic structures a lower bound for this splitting exists. We underpin our numerical calculations by an insightful symmetry analysis.