Self-efficacy has been shown to be related to sick leave and to be a predictor of return to work after sickness absence. The aim of this study was to investigate whether factors related to sick leave predict self-efficacy in women on long-term sick leave because of pain and/or mental illness. This cross-sectional study uses baseline data from 337 Swedish women with pain and/or mental illness. All included women took part in vocational rehabilitation. Data were collected through a sick leave register and a baseline questionnaire. General self-efficacy, sociodemographics, self-rated health, anxiety, depression, view of the future, and social support were measured and analyzed by univariate and multivariate linear regression analyses. The full multivariate linear regression model, which included mental health factors together with all measured factors, showed that anxiety and depression were the only predictive factors of lower self-efficacy (adjusted R2=0.46, P<0.001) and explained 46% of the variance in self-efficacy. The mean scores of general self-efficacy were low, especially in women born abroad, those with low motivation, those with uncertainties about returning to work, and women reporting distrust. Anxiety and depression are important factors to consider when targeting self-efficacy in vocational rehabilitation.