We report on a proof-of-principle demonstration of a two-stage cascaded optical inverse free-electron laser (IFEL) accelerator in which an electron beam is accelerated by a strong laser pulse after being packed into optical microbunches by a weaker initial laser pulse. We show experimentally that injection of precisely prepared optical microbunches into an IFEL allows net acceleration or deceleration of the beam, depending on the relative phase of the two laser pulses. The experimental results are in excellent agreement with simulation. The demonstrated technique holds great promise to significantly improve the beam quality of IFELs and may have a strong impact on emerging laser accelerators driven by high-power optical lasers.