Two-color terahertz (THz) generation is a field-matter process combining an optical pulse and its second harmonic. Its application in condensed matter is challenged by the lack of phase matching among multiple interacting fields. Here, we demonstrate phase-matching-free two-color THz conversion in condensed matter by introducing a highly resonant absorptive system. The generation is driven by a third-order nonlinear interaction localized at the surface of a narrow-band-gap semiconductor, and depends directly on the relative phase between the two colors. We show how to isolate the third-order effect among other competitive THz-emitting surface mechanisms, exposing the general features of the two-color process.