The aim of this study is to describe the prevalence of fractures in men with rheumatoid arthritis (RA) and identify potential risk factors associated with skeletal fragility. We consecutively studied 50 men with RA. Clinical risk factors were evaluated by clinical questionnaire, functional capacity by M-HAQ1, and disease activity by DAS-28. RA men were compared to 52 healthy controls paired for age and BMI. Bone mineral density (BMD) and quantitative ultrasound (QUS) at the heel were performed in all participants. Morphometric vertebral fractures (VF) were classified by a semiquantitative method. Men with RA were 51.7 years old on average and had mean disease duration of 115 months. Fragility fractures were found in 40% of individuals, of which 36% were VF, significantly higher than in healthy controls (p < 0.01). Age, anthropometric data, and lifestyle were similar between RA men with and without fractures. About 94% of the men with RA were on long-term glucocorticoid (GC) use. Patients with fractures were more frequently positive for rheumatoid factor (RF), had longer morning stiffness, and higher DAS-28 when compared to patients without fractures (p ≤ 0.05). In addition, they had significantly lower spine and hip BMD as well as a lower stiffness index (p ≤ 0.05). There was no statistically significant correlation between fracture and cumulative GC use. The final model of logistic regression showed a significant association and interaction between lower weight and physical activity in men with RA and fragility fractures. RA in men as well as in women is a risk factor for fragility fractures. The risk of fractures is higher in patients with positive RF, prolonged morning stiffness, higher scores of disease activity, and lower values of BMD and QUS.