The intratumoral microenvironment, or stroma, is of major importance in the pathobiology of pancreatic ductal adenocarcinoma (PDA), and specific conditions in the stroma may promote increased cancer aggressiveness. We hypothesized that this heterogeneous and evolving compartment drastically influences tumor cell abilities, which in turn influences PDA aggressiveness through crosstalk that is mediated by extracellular vesicles (EVs). Here, we have analyzed the PDA proteomic stromal signature and identified a contribution of the annexin A6/LDL receptor-related protein 1/thrombospondin 1 (ANXA6/LRP1/TSP1) complex in tumor cell crosstalk. Formation of the ANXA6/LRP1/TSP1 complex was restricted to cancer-associated fibroblasts (CAFs) and required physiopathologic culture conditions that improved tumor cell survival and migration. Increased PDA aggressiveness was dependent on tumor cell-mediated uptake of CAF-derived ANXA6+ EVs carrying the ANXA6/LRP1/TSP1 complex. Depletion of ANXA6 in CAFs impaired complex formation and subsequently impaired PDA and metastasis occurrence, while injection of CAF-derived ANXA6+ EVs enhanced tumorigenesis. We found that the presence of ANXA6+ EVs in serum was restricted to PDA patients and represents a potential biomarker for PDA grade. These findings suggest that CAF-tumor cell crosstalk supported by ANXA6+ EVs is predictive of PDA aggressiveness, highlighting a therapeutic target and potential biomarker for PDA.