As a critical enabler for beyond fifth-generation (B5G) technology, millimeter wave (mmWave) beamforming for mmWave has been studied for many years. Multi-input multi-output (MIMO) system, which is the baseline for beamforming operation, rely heavily on multiple antennas to stream data in mmWave wireless communication systems. High-speed mmWave applications face challenges such as blockage and latency overhead. In addition, the efficiency of the mobile systems is severely impacted by the high training overhead required to discover the best beamforming vectors in large antenna array mmWave systems. In order to mitigate the stated challenges, in this paper, we propose a novel deep reinforcement learning (DRL) based coordinated beamforming scheme where multiple base stations serve one mobile station (MS) jointly. The constructed solution then uses a proposed DRL model and predicts the suboptimal beamforming vectors at the base stations (BSs) out of possible beamforming codebook candidates. This solution enables a complete system that facilitates highly mobile mmWave applications with dependable coverage, minimal training overhead, and low latency. Numerical results demonstrate that our proposed algorithm remarkably increases the achievable sum rate capacity for the highly mobile mmWave massive MIMO scenario while ensuring low training and latency overhead.
Keywords: beamforming; deep reinforcement learning; massive MIMO; mmWave; vehicular network.