Lensless coherent diffractive imaging usually requires iterative phase-retrieval for recovering the missing phase information. Holographic techniques, such as Fourier-transform holography (FTH) or holography with extended references (HERALDO), directly provide this phase information and thus allow for a direct non-iterative reconstruction of the sample. In this paper, we analyze the effect of detector noise on the reconstruction for FTH and HERALDO with linear and rectangular references. We find that HERALDO is more sensitive to this type of noise than FTH, especially if rectangular references are employed. This excessive noise, caused by the necessary differentiation step(s) during reconstruction in case of HERALDO, additionally depends on the numerical implementation. When considering both shot-noise and detector noise, we find that FTH provides a better signal-to-noise ratio (SNR) than HERALDO if the available photon flux from the light source is low. In contrast, at high photon flux HERALDO provides better SNR and resolution than FTH. Our findings will help in designing optimum holographic imaging experiments particularly in the photon-flux-limited regime where most ultrafast experiments operate.