Nonlinear optical responses provide a powerful way to understand the microscopic interactions between laser fields and matter. They are critical for plenty of applications, such as in lasers, integrated photonic circuits, biosensing and medical tools. However, most materials exhibit weak optical nonlinearities or long response times when they interact with intense optical fields. Here, we strongly couple the exciton of cyanine dye J-aggregates to an optical mode of a Fabry-Perot (FP) cavity, and achieve an enhancement of the complex nonlinear refractive index by two orders of magnitude compared with that of the uncoupled condition. Moreover, the coupled system shows an ultrafast response of ~120 fs that we extract from optical cross-correlation measurements. The ultrafast and large enhancement of the optical nonlinar coefficients in this work paves the way for exploring strong coupling effects on various third-order nonlinear optical phenomena and for technological applications.