In front-line treatment of diffuse large B-cell lymphoma (DLBCL), prior studies suggest that concordant but not discordant involvement of the bone marrow (BM) portends a poor prognosis. The prognostic impact of bone marrow infiltration (BMI) in recurrent or refractory DLBCL (r/rDLBCL) and transformed indolent lymphoma (r/rTRIL) patients is less clear. Thus, we examined the prognostic significance of the infiltration of bone marrow (BMI) by concordant, large B-cells (conBMI) and discordant, small B-cells (disBMI) in this patient group. We performed a single center retrospective analysis of the prognostic impact of BMI diagnosed before start of second-line treatment as well as multiple clinicopathologic variables in 82 patients with r/rDLBCL or r/rTRIL intended to treat with autologous SCT. Twenty-five of 82 patients (30.5%) had BMI. Out of these, 19 (76%) had conBMI and 6 (24%) had disBMI. In patients with conBMI but not disBMI, uni- and multivariate analysis revealed inferior progression free survival (PFS) and overall survival (OS) compared to patients without BMI (median PFS, 9.2 vs 17.45 months, log rank: p = 0.049; Hazard Ratio, 2.34 (Confidence Interval, 1.24-4.44), p = 0.009; median OS 14.72 vs 28.91 months, log rank: p = 0.017; Hazard Ratio, 2.76 (Confidence Interval, 1.43-5.31), p = 0.002). ConBMI was strongly associated with nonGCB subtype as classified by the Hans algorithm (82.4% vs 17.6%, p = 0.01). ConBMI comprised an independent predictor of poor prognosis in primary and secondary r/rDLBCL. Incorporating conBMI in the pretherapeutic risk assessment for r/rDLBCL and r/rTRIL patients may be useful for prognostication, for stratification in clinical trials, and to assess new therapies for this high-risk patient subset that might not benefit from SCT in second-line treatment.