We have successfully demonstrated all-optical wavelength conversion of a 640-Gbit/s line-rate return-to-zero differential phase-shift keying (RZ-DPSK) signal based on low-power four wave mixing (FWM) in a silicon photonic chip with a switching energy of only ~110 fJ/bit. The waveguide dispersion of the silicon nanowire is nano-engineered to optimize phase matching for FWM and the switching power used for the signal processing is low enough to reduce nonlinear absorption from two-photon-absorption (TPA). These results demonstrate that high-speed wavelength conversion is achievable in silicon chips with high data integrity and indicate that high-speed operation can be obtained at moderate power levels where nonlinear absorption due to TPA and free-carrier absorption (FCA) is not detrimental. This demonstration can potentially enable high-speed optical networks on a silicon photonic chip.