Accumulating data indicates that tumor-derived extracellular vesicles (EVs) are responsible for tumor-promoting effects. However, if tumor EVs also prepare the tumor-bearing organ for subsequent tumor growth, and if this effect is different in low and high malignant tumors is not thoroughly explored. Here we used orthotopic rat Dunning R-3327 prostate tumors to compare the role of EVs from fast growing and metastatic MatLyLu (MLL) tumors with EVs from more indolent and non-metastatic Dunning G (G) tumors. Prostate tissue pre-conditioned with MLL-EVs in vivo facilitated G tumor establishment compared to G-EVs. MLL-EVs increased prostate epithelial proliferation and macrophage infiltration into the prostate compared to G-EVs. Both types of EVs increased macrophage endocytosis and the mRNA expression of genes associated with M2 polarization in vitro, with MLL-EVs giving the most pronounced effects. MLL-EVs also altered the mRNA expression of growth factors and cytokines in primary rat prostate fibroblasts compared to G-EVs, suggesting fibroblast activation. Our findings propose that EVs from metastatic tumors have the ability to prime the prostate tissue and enhance tumor growth to a higher extent than EVs from non-metastatic tumors. Identifying these differences could lead to novel therapeutic targets and potential prognostic markers for prostate cancer.