Holographic fluorescence imaging is very promising, as it can obtain three-dimensional fluorescence imaging without scanning. However, the current method usually records holograms far from the image plane, with the fluorescence decaying when spreading broadly. Here we show that the signal-to-noise ratio (SNR) of fluorescence holography can be improved by recording the high-contrast interferogram near the image plane. We found that this can be achieved by setting the focal length of the lens for the reference wave (f(2)) close to that for the object wave (f(1)). With experiments, we demonstrate an example of an increase of about 21 times in SNR by changing f(2) from infinity to 226 mm, which is close to f(1) (323 mm).