Novel double-layer bidirectional LSTM network with improved attention mechanism for predicting energy consumption

Abstract

For power generation management and power system dispatching, it is of big significance to predict the consumption of electric energy accurately. For the sake of improving the prediction accuracy of power consumption, taking the complex features of time series data into consideration, a novel neural network sandwich structure with an improved attention mechanism is inserted into the double-layer bidirectional long short-term memory network shortened as A-DBLSTM is put forward in this article. In A-DBLSTM, compared with traditional attention mechanism, the presented attention mechanism focuses on different features in each time unit and the A-DBLLSTM network extracts time information in sequence. The parameter optimization of A-DBLSTM is based on the method of particle swarm optimization (PSO). For confirming the effectiveness and feasibility of A-DBLSTM, case studies using two datasets of the hourly temperature values and power loads between 2012 and 2014 and the electric energy consumption are carried out. The experimental results indicate that the presented A-DBLSTM with the novel sandwich network structure achieves superior performance in the aspects of the mean square error, root mean square, the average absolute error and the mean absolute percentage error to other advanced methods. What is more, the factors that have the greatest impact on the prediction performance can be found through analyzing the heatmap of the attention layer.

Keywords: Attention mechanism; Bi-LSTM; Energy consumption; Particle swarm optimization; Time series.