The important role of angiogenesis in cancer development has driven many researchers to investigate the prospects of noninvasive cancer diagnosis based on the technology of contrast-enhanced ultrasound (CEUS) imaging. This paper presents a deep learning framework to detect prostate cancer in the sequential CEUS images. The proposed method uniformly extracts features from both the spatial and the temporal dimensions by performing three-dimensional convolution operations, which captures the dynamic information of the perfusion process encoded in multiple adjacent frames for prostate cancer detection. The deep learning models were trained and validated against expert delineations over the CEUS images recorded using two types of contrast agents, i.e., the anti-PSMA based agent targeted to prostate cancer cells and the non-targeted blank agent. Experiments showed that the deep learning method achieved over 91 percent specificity and 90 percent average accuracy over the targeted CEUS images for prostate cancer detection, which was superior ( ) than previously reported approaches and implementations.