Subsets of breast tumors present major clinical challenges, including triple-negative, metastatic/recurrent disease and rare histologies. Here, we developed 37 patient-derived xenografts (PDX) from these difficult-to-treat cancers to interrogate their molecular composition and functional biology. Whole-genome and transcriptome sequencing and reverse-phase protein arrays revealed that PDXs conserve the molecular landscape of their corresponding patient tumors. Metastatic potential varied between PDXs, where low-penetrance lung micrometastases were most common, though a subset of models displayed high rates of dissemination in organotropic or diffuse patterns consistent with what was observed clinically. Chemosensitivity profiling was performed in vivo with standard-of-care agents, where multi-drug chemoresistance was retained upon xenotransplantation. Consolidating chemogenomic data identified actionable features in the majority of PDXs, and marked regressions were observed in a subset that was evaluated in vivo. Together, this clinically-annotated PDX library with comprehensive molecular and phenotypic profiling serves as a resource for preclinical studies on difficult-to-treat breast tumors.