We present an experiment using a sample of laser-cooled Rb atoms to show that cross-phase modulation schemes continue to benefit from electromagnetically induced transparency (EIT) even as the transparency window is made narrower than the signal bandwidth (i.e., for signal pulses much shorter than the response time of the EIT system). Addressing concerns that narrow EIT windows might not prove useful for such applications, we show that while the peak phase shift saturates in this regime, it does not drop, and the time-integrated effect continues to scale inversely with EIT window width. This integrated phase shift is an important figure of merit for tasks such as the detection of single-photon-induced cross-phase shifts. Only when the window width approaches the system's dephasing rate γ does the peak phase shift begin to decrease, leading to an integrated phase shift that peaks when the window width is equal to 4γ.