The dynamics of the motile magnetotactic bacterium Magnetospirillum gryphiswaldense in a rotating magnetic field is investigated experimentally and analyzed by a theoretical model. These elongated bacteria are propelled by single flagella at each bacterial end and contain a magnetic filament formed by a linear assembly of approximately 40 ferromagnetic nanoparticles. The movements of the bacteria in suspension are analyzed by consideration of the orientation of their magnetic dipoles in the field, the hydrodynamic resistance of the bacteria, and the propulsive force of the flagella. Several novel features found in experiments include a velocity reversal during motion in the rotating field and an interesting diffusive wandering of the trajectory curvature centers. A new method to measure the magnetic moment of an individual bacterium is proposed based on the theory developed.