The negatively charged silicon vacancy (SiV) color center in diamond has recently proven its suitability for bright and stable single photon emission. However, its electronic structure so far has remained elusive. We here explore the electronic structure by exposing single SiV defects to a magnetic field where the Zeeman effect lifts the degeneracy of magnetic sublevels. The similar responses of single centers and a SiV ensemble in a low strain reference sample prove our ability to fabricate almost perfect single SiVs, revealing the true nature of the defect's electronic properties. We model the electronic states using a group-theoretical approach yielding a good agreement with the experimental observations. Furthermore, the model correctly predicts polarization measurements on single SiV centers and explains recently discovered spin selective excitation of SiV defects.