The effects of four heteroatoms (B, N, P, and S) with three doping patterns on graphene quantum dots (GQDs) are systematically investigated using time-dependent density functional theory (TD-DFT). The absorption spectra and HOMO-LUMO gaps are quantitatively analyzed to study the correlations between the optical properties and heteroatom doping of doped GQDs. Heteroatom doping can endow GQDs with various new optical and structural properties, depending on the dopants and doping configurations. Compared with the absorption spectra of pristine GQD, both N and S surface doping demonstrate a slight blue shift, whereas B and P doping lead to a blue shift for edge-doped GQDs with heteroatoms in a pentatomic ring. The absorption process is investigated along with excited state analysis, which includes the density of state, natural transition orbital, and charge difference density. The results indicate that large radius atoms assist charge transfer in the excited state and play an important role in recombining the electron density distribution in the doped GQDs.