The force field of optical tweezers is commonly assumed to be conservative, neglecting the complex action of the scattering force. Using a novel method that extracts local forces from trajectories of an optically trapped particle, we measure the three-dimensional force field experienced by a Rayleigh particle with 10 nm spatial resolution and femtonewton precision in force. We find that the force field is nonconservative with the nonconservative component increasing radially away from the optical axis, in agreement with the Gaussian beam model of the optical trap.