Two-Stage Multiarmed Bandit for Reconfigurable Intelligent Surface Aided Millimeter Wave Communications

Ehab Mahmoud Mohamed; Sherief Hashima; Kohei Hatano; Saud Alhajaj Aldossari

doi:10.3390/s22062179

Two-Stage Multiarmed Bandit for Reconfigurable Intelligent Surface Aided Millimeter Wave Communications

Sensors (Basel). 2022 Mar 10;22(6):2179. doi: 10.3390/s22062179.

Authors

Ehab Mahmoud Mohamed^{1

2}, Sherief Hashima^{3

4}, Kohei Hatano^{3

5}, Saud Alhajaj Aldossari¹

Affiliations

¹ Electrical Engineering Department, College of Engineering at Wadi Addwasir, Prince Sattam Bin Abdulaziz University, Wadi Addwasir 11991, Saudi Arabia.
² Electrical Engineering Department, Faculty of Engineering, Aswan University, Aswan 81542, Egypt.
³ Computational Learning Theory Team, RIKEN-Advanced Intelligent Project, Fukuoka 819-0395, Japan.
⁴ Engineering and Scientific Equipment's Department, Egyptian Atomic Energy Authority, Cairo 13759, Egypt.
⁵ Faculty of Arts and Science, Kyushu University, Fukuoka 819-0395, Japan.

Abstract

A reconfigurable intelligent surface (RIS) is a promising technology that can extend short-range millimeter wave (mmWave) communications coverage. However, phase shifts (PSs) of both mmWave transmitter (TX) and RIS antenna elements need to be optimally adjusted to effectively cover a mmWave user. This paper proposes codebook-based phase shifters for mmWave TX and RIS to overcome the difficulty of estimating their mmWave channel state information (CSI). Moreover, to adjust the PSs of both, an online learning approach in the form of a multiarmed bandit (MAB) game is suggested, where a nested two-stage stochastic MAB strategy is proposed. In the proposed strategy, the PS vector of the mmWave TX is adjusted in the first MAB stage. Based on it, the PS vector of the RIS is calibrated in the second stage and vice versa over the time horizon. Hence, we leverage and implement two standard MAB algorithms, namely Thompson sampling (TS) and upper confidence bound (UCB). Simulation results confirm the superior performance of the proposed nested two-stage MAB strategy; in particular, the nested two-stage TS nearly matches the optimal performance.

Keywords: Thompson sampling; millimeter wave; multiarmed bandit; reconfigurable intelligent surface; upper confidence bound.

Grants and funding

(IF-PSAU-2021/01/18041)/Deputyship for Research & Innovation, Ministry of Education in Saudi Arabia for funding this research work through the project number (IF-PSAU-2021/01/18041)