Background Diffusion-weighted (DW) imaging is useful in detecting tumor in the primary tumor bed in locally advanced rectal cancer (LARC) after neoadjuvant therapy, but its value in detecting extramural venous invasion (EMVI) and tumor deposit is not well validated. Purpose To evaluate diagnostic accuracy and association with patient prognosis of viable EMVI and tumor deposit on DW images in patients with LARC after neoadjuvant therapy using whole-mount pathology specimens. Materials and Methods This retrospective study included patients who underwent neoadjuvant therapy and surgery from 2018 to 2021. Innovative five-point Likert scale was used by two radiologists to independently evaluate the likelihood of viable EMVI and tumor deposit on restaging DW MRI scans in four axial quadrants (12 to 3 o'clock, 3 to 6 o'clock, 6 to 9 o'clock, and 9 to 12 o'clock). Diagnostic accuracy was assessed at both the per-quadrant and per-patient level, with whole-mount pathology as the reference standard. Weighted κ values for interreader agreement and Cox regression models for disease-free survival and overall survival analyses were used. Results A total of 117 patients (mean age, 56 years ± 12 [SD]; 70 male, 47 female) were included. Pathologically proven viable EMVI and tumor deposit was detected in 29 of 117 patients (25%) and in 44 of 468 quadrants (9.4%). Per-quadrant analyses showed an area under the receiver operating characteristics curve of 0.75 (95% CI: 0.68, 0.83), with sensitivity and specificity of 55% and 96%, respectively. Good interreader agreement was observed between the radiologists (κ = 0.62). Per-patient analysis showed sensitivity and specificity of 62% and 93%, respectively. The presence of EMVI and tumor deposit on restaging DW MRI scans was associated with worse disease-free survival (hazard ratio [HR], 5.6; 95% CI: 2.4, 13.3) and overall survival (HR, 8.9; 95% CI: 1.6, 48.5). Conclusion DW imaging using the five-point Likert scale showed high specificity and moderate sensitivity in the detection of viable extramural venous invasion and tumor deposits in LARC after neoadjuvant therapy, and its presence on restaging DW MRI scans is associated with worse prognosis. Published under a CC BY 4.0 license. Supplemental material is available for this article. See also the editorial by Méndez and Ayuso in this issue.