Iron oxide cores of 35 nm are coated with gold nanoparticles so that individual particle motion can be tracked in real time through the plasmonic response using dark field optical microscopy. Although Brownian and viscous drag forces are pronounced for nanoparticles, we show that magnetic manipulation is possible using large magnetic field gradients. The trajectories are analyzed to separate contributions from the different types of forces. With field gradients up to 3000 T/m, forces as small as 1.5 fN are detected.