The interaction between a confined electron and the nuclei of an optically active quantum dot provides a uniquely rich manifestation of the central spin problem. Coherent qubit control combines with an ultrafast spin-photon interface to make these confined spins attractive candidates for quantum optical networks. Reaching the full potential of spin coherence has been hindered by the lack of knowledge of the key irreversible environment dynamics. Through all-optical Hahn echo decoupling we now recover the intrinsic coherence time set by the interaction with the inhomogeneously strained nuclear bath. The high-frequency nuclear dynamics are directly imprinted on the electron spin coherence, resulting in a dramatic jump of coherence times from few tens of nanoseconds to the microsecond regime between 2 and 3 T magnetic field and an exponential decay of coherence at high fields. These results reveal spin coherence can be improved by applying large magnetic fields and reducing strain inhomogeneity.