The timely detection of equipment failure can effectively avoid industrial safety accidents. The existing equipment fault diagnosis methods based on single-mode signal not only have low accuracy, but also have the inherent risk of being misled by signal noise. In this paper, we reveal the possibility of using multi-modal monitoring data to improve the accuracy of equipment fault prediction. The main challenge of multi-modal data fusion is how to effectively fuse multi-modal data to improve the accuracy of fault prediction. We propose a multi-modal learning framework for fusion of low-quality monitoring data and high-quality monitoring data. In essence, low-quality monitoring data are used as a compensation for high-quality monitoring data. Firstly, the low-quality monitoring data is optimized, and then the features are extracted. At the same time, the high-quality monitoring data is dealt with by a low complexity convolutional neural network. Moreover, the robustness of the multi-modal learning algorithm is guaranteed by adding noise to the high-quality monitoring data. Finally, different dimensional features are projected into a common space to obtain accurate fault sample classification. Experimental results and performance analysis confirm the superiority of the proposed algorithm. Compared with the traditional feature concatenation method, the prediction accuracy of the proposed multi-modal learning algorithm can be improved by up to 7.42%.
Keywords: Internet of Things; convolutional neural network; equipment fault prediction; multi-modal learning.