We investigated the atomic geometry, electronic band structure, and optical absorption of nitrogen hyperdoped silicon based on first-principles calculations. The results show that all the paired nitrogen defects we studied do not introduce intermediate band, while most of single nitrogen defects can introduce intermediate band in the gap. Considering the stability of the single defects and the rapid resolidification following the laser melting process in our sample preparation method, we conclude that the substitutional nitrogen defect, whose fraction was tiny and could be neglected before, should have considerable fraction in the hyperdoped silicon and results in the visible sub-band-gap absorption as observed in the experiment. Furthermore, our calculations show that the substitutional nitrogen defect has good stability, which could be one of the reasons why the sub-band-gap absorptance remains almost unchanged after annealing.