The nonzero response time of the Kerr [chi(3)] nonlinearity determines the quantum-limited noise figure of chi(3) parametric amplifiers. This nonzero response time of the nonlinearity requires coupling of the parametric amplification process to a molecular-vibration phonon bath, causing the addition of excess noise through Raman gain or loss at temperatures above 0 K. The effect of this excess noise on the noise figure can be surprisingly significant. We derive analytical expressions for this quantum-limited noise figure for phase-insensitive operation of a chi(3) amplifier and show good agreement with published noise-figure measurements.