Ferroelectric materials for photovoltaics have sparked great interest because of their switchable photoelectric responses and above-bandgap photovoltages that violate conventional photovoltaic theory. However, their relatively low photocurrent and power conversion efficiency limit their potential application in solar cells. To improve performance, conventional strategies focus mainly on narrowing the bandgap to better match the solar spectrum, leaving the fundamental connection between polar order and photovoltaic effect largely overlooked. We report large photovoltaic enhancement by A-site substitutions in a model ferroelectric photovoltaic material, BiFeO3. As revealed by optical measurements and supported by theoretical calculations, the enhancement is accompanied by the chemically driven rotational instability of the polarization, which, in turn, affects the charge transfer at the band edges and drives a direct-to-indirect bandgap transition, highlighting the strong coupling between polarization, lattice, and orbital order parameters in ferroelectrics. Polar order engineering thus provides an additional degree of freedom to further boost photovoltaic efficiency in ferroelectrics and related materials.