Survival analysis is a valid solution for cancer treatments and outcome evaluations. Due to the wide application of medical imaging and genome technology, computer-aided survival analysis has become a popular and promising area, from which we can get relatively satisfactory results. Although there are already some impressive technologies in this field, most of them make some recommendations using single-source medical data and have not combined multi-level and multi-source data efficiently. In this paper, we propose a novel pathological images and gene expression data fusion framework to perform the survival prediction. Different from previous methods, our framework can extract correlated multi-scale deep features from whole slide images (WSIs) and dimensionality reduced gene expression data respectively for jointly survival analysis. The experiment results demonstrate that the integrated multi-level image and genome features can achieve higher prediction accuracy compared with single-source features.