Market Model for Resource Allocation in Emerging Sensor Networks with Reinforcement Learning

Yue Zhang; Bin Song; Ying Zhang; Xiaojiang Du; Mohsen Guizani

doi:10.3390/s16122021

Market Model for Resource Allocation in Emerging Sensor Networks with Reinforcement Learning

Sensors (Basel). 2016 Nov 29;16(12):2021. doi: 10.3390/s16122021.

Authors

Yue Zhang¹, Bin Song², Ying Zhang³, Xiaojiang Du⁴, Mohsen Guizani⁵

Affiliations

¹ The State Key Laboratory of Integrated Services Networks, Xidian University, Xi'an 710071, China. y.zhang@stu.xidian.edu.cn.
² The State Key Laboratory of Integrated Services Networks, Xidian University, Xi'an 710071, China. bsong@mail.xidian.edu.cn.
³ The State Key Laboratory of Integrated Services Networks, Xidian University, Xi'an 710071, China. yzxdedm@163.com.
⁴ Department of Computer and Information Sciences, Temple University, Philadelphia, PA 19122, USA. dxj@ieee.org.
⁵ Department of Electrical and Computer Engineering, University of Idaho, Moscow, ID 83844, USA. mguizani@ieee.org.

Abstract

Emerging sensor networks (ESNs) are an inevitable trend with the development of the Internet of Things (IoT), and intend to connect almost every intelligent device. Therefore, it is critical to study resource allocation in such an environment, due to the concern of efficiency, especially when resources are limited. By viewing ESNs as multi-agent environments, we model them with an agent-based modelling (ABM) method and deal with resource allocation problems with market models, after describing users' patterns. Reinforcement learning methods are introduced to estimate users' patterns and verify the outcomes in our market models. Experimental results show the efficiency of our methods, which are also capable of guiding topology management.

Keywords: Internet of Things; agent-based modelling; emerging sensor networks; market model; reinforcement learning; resource allocation; topology management.