All-optical switching (AOS) or cross-phase modulation (XPM) based on the effect of electromagnetically induced transparency (EIT) makes one photon switched or phase-modulated by another possible. The existence of four-wave mixing (FWM) process greatly diminishes the switching or phase-modulation efficiency and hinders the single-photon operation. We proposed and experimentally demonstrated an idea that with an optimum detuning the EIT-based AOS can be completely intact even under the influence of FWM. The results of the work can be directly applied to the EIT-based XPM. Our work makes the AOS and XPM schemes more flexible and the single-photon operation possible in FWM-allowed systems.