This Letter describes a high-speed yet simple fiber-wireless-fiber system in the 100-GHz band using a photonics-enabled receiver and optical phase modulator. At the antenna site, a millimeter-wave signal is down-converted to the microwave band using an electronic mixer with a local oscillator signal generated remotely using photonic technology. The down-converted signal is converted to an optical signal using an optical phase modulator. At the receiver, a simple direct detection of the phase-modulated signal is performed using optical filtering technology. To demonstrate the proof-of-concept, we successfully transmitted a 64-quadrature amplitude modulation orthogonal frequency-division multiplexing signal with a record line rate of 80 Gb/s (net data rate of 55 Gb/s) over a system consisting of two radio-over-fiber links and a 5 m wireless link in the 100-GHz band. The proposed system with simple antenna sites and optical transceivers can facilitate the deployment of ultra-dense small cells in high-frequency bands in beyond-5G networks.