This paper proposes a novel hybrid beamforming and relay selection scheme for spectral efficiency maximization in a non-regenerative multi-relay multi-input multi-output (MIMO) system. The analog beamforming component in the radio-frequency (RF) domain must follow an element-wise constant modulus constraint, which makes the underlying design problem mathematically intractable and therefore, it is quite challenging to obtain the global optimal solution. To address this problem, phase-only precoding/combining matrices are derived by maximizing the end-to-end received signal-to-noise ratio (SNR) under transmit power constraint at the source and each relay node. This task is achieved by decomposing the original complicated optimization problem into two independent components. The first component designs the RF precoder/combiner at source and relay nodes by maximizing the received SNR at relay nodes. While the second component attempts to derive the analog precoder/combiner at relay nodes and destination by maximizing the received SNR at the destination. Digital baseband processing matrices are obtained by deriving the closed-form expression, which minimizes interference among different sub-channels. Finally, the relay selection is made by maximizing the overall SNR from the source to the destination. Computer simulations reveal that the performance of the proposed algorithm is close to its fully digital counterpart and approximately 6% higher than the specified relay-assisted hybrid beamforming techniques. Moreover, the proposed method achieves more than 15% higher performance in a sparse scattering environment when compared with the given relay selection techniques.
Keywords: MIMO; hybrid beamforming; millimeter wave; non-regenerative relay.