We consider reconstruction of signals by a direct method for the solution of the discrete Fourier system. We note that the reconstruction of a time-limited signal can be simply realized by using only either the real part or the imaginary part of the discrete Fourier transform (DFT) matrix. Therefore, based on the study of the special structure of the real and imaginary parts of the discrete Fourier matrix, we propose a fast direct method for the signal reconstruction problem, which utilizes the numerically truncated singular value decomposition. The method enables us to recover the original signal in a stable way from the frequency information, which may be corrupted by noise and/or some missing data. The classical inverse Fourier transform cannot be applied directly in the latter situation. The pivotal point of the reconstruction is the explicit computation of the singular value decomposition of the real part of the DFT for any order. Numerical experiments for 1D and 2D signal reconstruction and image restoration are given.