The extent to which medical oncologists consider non-clinical patient factors when deciding to recommend adjuvant chemotherapy is unknown. Medical oncologists who treated a population-based sample of early stage breast cancer patients reported to the Los Angeles and Detroit Surveillance, Epidemiology, and End Results registries 2005-2007 were asked how strongly they consider a patient's ability to follow instructions, level of social support, and level of work support/flexibility in decisions to recommend adjuvant chemotherapy. Responses of 4 (Quite strongly) or 5 (Very strongly) on a five-point Likert scale defined strong consideration. Associations between oncologist/practice characteristics and strong consideration of each non-clinical factor were examined. 134 oncologists (66 %) reported strong consideration of one or more factor. Ability to follow instructions was strongly considered by 120 oncologists (59 %), social support by 78 (38 %), and work support/flexibility by 73 (36 %). Larger percent of practice devoted to breast cancer was associated with lower likelihood of strongly considering ability to follow instructions [odds ratio (OR) 0.98, 95 % confidence interval (CI) 0.97-0.99; P = 0.04]. Increased years in practice was associated with lower likelihood of strongly considering social support (OR 0.96, CI 0.93-0.99; P = 0.011), while non-white race (OR 2.1, CI 1.03-4.26; P = 0.041) and tumor board access (OR 2.04, CI 1.01-4.12; P = 0.048) were associated with higher likelihood. Non-white race was associated with strongly considering work support/flexibility (OR 2.44, CI 1.21-4.92; P = 0.013). Tumor board access (OR 2, CI 1.00-4.02; P = 0.051) was borderline significant. Non-clinical patient factors play a role in medical oncologist decision-making for breast cancer adjuvant chemotherapy recommendations.