Prostate cancer (PCa) is a prevalent malignancy affecting men worldwide, and plant-based diets have been widely advocated for their health benefits. The aim of this study was to test the association between general, healthy, and unhealthy pro-vegetarian plant-based diets and PCa severity on 118 consecutive patients undergoing prostatectomy in a university hospital in Italy. Food frequency questionnaires were used to calculate scores for dietary patterns. Multivariate logistic regression analyses were used to calculate odds ratios (ORs) and 95% confidence intervals (CIs) to test the associations. A general plant-based diet was not associated with cancer severity, while patients reporting a higher adherence to a healthy plant-based diet were less likely to have a more severe PCa (for medium/high vs. low-risk PCa, OR = 0.27, 95% CI: 0.08, 0.88; for high vs. medium/low-risk PCa, OR = 0.09, 95% CI: 0.02, 0.39). Patients resulting in higher adherence to an unhealthy plant-based diet were more likely to be diagnosed with more severe PCa (OR = 6.15, 95% CI: 1.70, 22.24). In conclusion, plant-based dietary patterns may have a different impact on PCa severity depending on the quality of the foods included.