We report that the addition of a non-photoactive tertiary polymer phase in the binary bulk heterojunction (BHJ) polymer solar cell leads to a self-assembled columnar nanostructure, enhancing the charge mobilities and photovoltaic efficiency with surprisingly increased optimal active blend thicknesses over 300 nm, 3-4 times larger than that of the binary counterpart. Using the prototypical poly(3-hexylthiophene) (P3HT):fullerene blend as a model BHJ system, we discover that the inert poly(methyl methacrylate) (PMMA) added in the binary BHJ blend self-assembles into vertical columns, which not only template the phase segregation of electron acceptor fullerenes but also induce the out-of-plane rotation of the edge-on-orientated crystalline P3HT phase. Using complementary interrogation methods including neutron reflectivity, X-ray scattering, atomic force microscopy, transmission electron microscopy, and molecular dynamics simulations, we show that the enhanced charge transport originates from the more randomized molecular stacking of the P3HT phase and the spontaneous segregation of fullerenes at the P3HT/PMMA interface, driven by the high surface tension between the two polymeric components. The results demonstrate a potential method for increasing the thicknesses of high-performance polymer BHJ solar cells with improved photovoltaic efficiency, alleviating the burden of stringently controlling the ultrathin blend thickness during the roll-to-roll-type large-area manufacturing environment.