Advantage distillation (AD) provides a means of separating highly correlated raw key bits from weakly correlated information in quantum key distribution (QKD). In this Letter, we apply the AD method to improve the performance of reference-frame-independent QKD (RFI-QKD). Simulation results show that, compared with RFI-QKD without AD, RFI-QKD with AD can tolerate higher system errors and obtain better performance on the secret key rate and transmission distance. Furthermore, we extend the AD method to RFI measurement-device-independent QKD (RFI-MDI-QKD) and demonstrate that the AD method can improve the performance of RFI-MDI-QKD more significantly.