Control over the spectral phase of a light pulse is a fundamental step toward arbitrary signal generation in a spectral band. For the terahertz spectral regime, pulse shaping holds the key for applications ranging from ultra-high speed wireless data transmission to quantum control with shaped fields. In this work, we demonstrate a technique for all-optical and reconfigurable control of the spectral phase of a light pulse in the important terahertz (THz) band. The technique is based on interaction of a guided THz pulse with patterned photoexcited regions within a uniform silicon-filled parallel-plate waveguide. We use this platform to demonstrate broadband and tunable positive and negative chirp of a THz pulse, as well as control of the pulse carrier envelope phase.