We describe a simple and robust approach for characterizing the spatially varying pupil aberrations of microscopy systems. In our demonstration with a standard microscope, we derive the location-dependent pupil transfer functions by first capturing multiple intensity images at different defocus settings. Next, a generalized pattern search algorithm is applied to recover the complex pupil functions at ~350 different spatial locations over the entire field-of-view. Parameter fitting transforms these pupil functions into accurate 2D aberration maps. We further demonstrate how these aberration maps can be applied in a phase-retrieval based microscopy setup to compensate for spatially varying aberrations and to achieve diffraction-limited performance over the entire field-of-view. We believe that this easy-to-use spatially-varying pupil characterization method may facilitate new optical imaging strategies for a variety of wide field-of-view imaging platforms.