DWSSA: Alleviating over-smoothness for deep Graph Neural Networks

Abstract

Graph Neural Networks (GNNs) have demonstrated great potential in achieving outstanding performance in various graph-related tasks, e.g., graph classification and link prediction. However, most of them suffer from the following issue: shallow networks capture very limited knowledge. Prior works design deep GNNs with more layers to solve the issue, which however introduces a new challenge, i.e., the infamous over-smoothness. Graph representation over emphasizes node features but only considers the static graph structure with a uniform weight are the key reasons for the over-smoothness issue. To alleviate the issue, this paper proposes a Dynamic Weighting Strategy (DWS) for addressing over-smoothness. We first employ Fuzzy C-Means (FCM) to cluster all nodes into several groups and get each node's fuzzy assignment, based on which a novel metric function is devised for dynamically adjusting the aggregation weights. This dynamic weighting strategy not only enables the intra-cluster interactions, but also inter-cluster aggregations, which well addresses undifferentiated aggregation caused by uniform weights. Based on DWS, we further design a Structure Augmentation (SA) step for addressing the issue of underutilizing the graph structure, where some potentially meaningful connections (i.e., edges) are added to the original graph structure via a parallelable KNN algorithm. In general, the optimized Dynamic Weighting Strategy with Structure Augmentation (DWSSA) alleviates over-smoothness by reducing noisy aggregations and utilizing topological knowledge. Extensive experiments on eleven homophilous or heterophilous graph benchmarks demonstrate the effectiveness of our proposed method DWSSA in alleviating over-smoothness and enhancing deep GNNs performance.

Keywords: Clustering; Deep graph neural networks; Node classification; Over-smoothness; Structure augmentation.