XPA, XPC-hHR23B, RPA, and TFIIH all are the damage recognition proteins essential for the early stage of nucleotide excision repair. Nonetheless, it is not clear how these proteins work together at the damaged DNA site. To get insight into the molecular mechanism of damage recognition, we carried out a comprehensive analysis on the interaction between damage recognition proteins and their assembly on damaged DNA. XPC physically interacted with XPA, but failed to stabilize the XPA-damaged DNA complex. Instead, XPC-hHR23B was effectively displaced from the damaged DNA by the combined action of RPA and XPA. A mutant RPA lacking the XPA interaction domain failed to displace XPC-hHR23B from damaged DNA, suggesting that XPA and RPA cooperate with each other to destabilize the XPC-hHR23B-damaged DNA complex. Interestingly, the presence of hHR23B significantly increased RPA/XPA-mediated displacement of XPC from damaged DNA, suggesting that hHR23B may modulate the binding of XPC to damaged DNA. Together, our results suggest that damage recognition occurs in a multistep process such that XPC-hHR23B initiates damage recognition, which was replaced by combined action of XPA and RPA. XPA and RPA, once forming a complex at the damage site, would likely work with TFIIH, XPG, and ERCC1-XPF for dual incision.