Highly-efficient optical generation of narrowband terahertz radiation enables unexplored technologies and sciences from compact electron acceleration to charge manipulation in solids. State-of-the-art conversion efficiencies are currently achieved using difference-frequency generation driven by temporal beating of chirped pulses but remain, however, far lower than desired or predicted. Here we show that high-order spectral phase fundamentally limits the efficiency of narrowband difference-frequency generation using chirped-pulse beating and resolve this limitation by introducing a novel technique based on tuning the relative spectral phase of the pulses. For optical terahertz generation, we demonstrate a 13-fold enhancement in conversion efficiency for 1%-bandwidth, 0.361 THz pulses, yielding a record energy of 0.6 mJ and exceeding previous optically-generated energies by over an order of magnitude. Our results prove the feasibility of millijoule-scale applications like terahertz-based electron accelerators and light sources and solve the long-standing problem of temporal irregularities in the pulse trains generated by interfering chirped pulses.