The charge separation yield at a bulk heterojunction sets the upper efficiency limit of an organic solar cell. Ultrafast charge transfer processes in polymer/fullerene blends have been intensively studied but much less is known about these processes in all-polymer systems. Here, we show that interfacial charge separation can occur through a polaron pair-derived hole transfer process in all-polymer photovoltaic blends, which is a fundamentally different mechanism compared to the exciton-dominated pathway in the polymer/fullerene blends. By utilizing ultrafast optical measurements, we have clearly identified an ultrafast hole transfer process with a lifetime of about 3 ps mediated by photo-excited polaron pairs which has a markedly high quantum efficiency of about 97%. Spectroscopic data show that excitons act as spectators during the efficient hole transfer process. Our findings suggest an alternative route to improve the efficiency of all-polymer solar devices by manipulating polaron pairs.