Background Preoperative response evaluation with neoadjuvant chemoradiotherapy remains a challenge in the setting of locally advanced rectal cancer. Recently, deep learning (DL) has been widely used in tumor diagnosis and treatment and has produced exciting results. Purpose To develop and validate a DL method to predict response of rectal cancer to neoadjuvant therapy based on diffusion kurtosis and T2-weighted MRI. Materials and Methods In this prospective study, participants with locally advanced rectal adenocarcinoma (≥cT3 or N+) proved at histopathology and baseline MRI who were scheduled to undergo preoperative chemoradiotherapy were enrolled from October 2015 to December 2017 and were chronologically divided into 308 training samples and 104 test samples. DL models were constructed primarily to predict pathologic complete response (pCR) and secondarily to assess tumor regression grade (TRG) (TRG0 and TRG1 vs TRG2 and TRG3) and T downstaging. Other analysis included comparisons of diffusion kurtosis MRI parameters and subjective evaluation by radiologists. Results A total of 383 participants (mean age, 57 years ± 10 [standard deviation]; 229 men) were evaluated (290 in the training cohort, 93 in the test cohort). The area under the receiver operating characteristic curve (AUC) was 0.99 for the pCR model in the test cohort, which was higher than the AUC for raters 1 and 2 (0.66 and 0.72, respectively; P < .001 for both). AUC for the DL model was 0.70 for TRG and 0.79 for T downstaging. AUC for pCR with the DL model was better than AUC for the best-performing diffusion kurtosis MRI parameters alone (diffusion coefficient in normal diffusion after correcting the non-Gaussian effect [Dapp value] before neoadjuvant therapy, AUC = 0.76). Subjective evaluation by radiologists yielded a higher error rate (1 - accuracy) (25 of 93 [26.9%] and 23 of 93 [24.8%] for raters 1 and 2, respectively) in predicting pCR than did evaluation with the DL model (two of 93 [2.2%]); the radiologists achieved a lower error rate (12 of 93 [12.9%] and 13 of 93 [14.0%] for raters 1 and 2, respectively) when assisted by the DL model. Conclusion A deep learning model based on diffusion kurtosis MRI showed good performance for predicting pathologic complete response and aided the radiologist in assessing response of locally advanced rectal cancer after neoadjuvant chemoradiotherapy. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Koh in this issue.