Accurate survival prediction is a critical goal in the prognosis of breast cancer patients because it can help physicians make more patient-friendly decisions and further guide appropriate treatment. Breast cancer is often caused by genetic abnormalities, which prompts researchers to consider information such as gene expression and copy number variation in addition to clinical data in their studies. The integration of these multi-modal data can improve the predictive power of models. However, with the highly unbalanced information of breast cancer patient data, it becomes a new challenge for breast cancer patient survival prediction to fully extract the characteristic information of these multi-modal data and to consider the complementarity of this information. To this end, we propose a deep multi-modal fusion network (DMMFN) to predict the five-year survival of breast cancer patients by integrating clinical data, copy number variation data, and gene expression data. The imbalanced dataset is first processed using the oversampling method SMOTE-NC. Then the abstract modal features of the multi-modal data are extracted by the two-layer one-dimensional convolutional neural network and the bi-directional long short-term memory network. Next, the weight coefficients of each modal data are dynamically adjusted using gated multimodal units to obtain fusion features. Finally, the fusion features are fed into the MaxoutMLP classifier to obtain the final prediction results. We conducted experiments on the METABRIC dataset to verify the validity of the multi-modal data and compared it with other methods. The comprehensive performance evaluation shows that DMMFN has better prediction performance.
Keywords: Multi-modal fusion; SMOTE-NC; breast cancer survival prediction; deep learning; gated multimodal unit.