The fluctuations in the conductance of graphene that arise from a long-range disorder potential induced by random impurities are investigated with an atomic tight-binding lattice. The screened impurities lead to a slow variation of the background potential and this varies the overall potential landscape as the Fermi energy or an applied magnetic field is varied. As a result, the phase interference varies randomly and leads to fluctuations in the conductance. Recently, experiments have shown that an applied magnetic field produces a remarkable reduction in the amplitude of these conductance fluctuations. We find qualitative agreement with these experiments, and it appears that the reduction in magnetic field of the fluctuations arises from a field induced smoothing of the conductance landscape.