Electromagnetic waves in the terahertz (THz) frequency range are widely used in spectroscopy, imaging and sensing. However, commercial, table-top systems covering the entire frequency range from 100 GHz to 10 THz are not available today. Fiber-coupled spectrometers, which employ photoconductive antennas as emitters and receivers, show a bandwidth limited to 6.5 THz and some suffer from spectral artifacts above 4 THz. For these systems, we identify THz absorption in the polar substrate of the photoconductive antenna as the main reason for these limitations. To overcome them, we developed photoconductive membrane (PCM) antennas, which consist of a 1.2 µm-thin InGaAs layer bonded on a Si substrate. These antennas combine efficient THz generation and detection in InGaAs with absorption-free THz transmission through a Si substrate. With these devices, we demonstrate a fiber-coupled THz spectrometer with a total bandwidth of 10 THz and an artifact-free spectrum up to 6 THz. The PCM antennas present a promising path toward fiber-coupled, ultrabroadband THz spectrometers.