Identification of intrinsic brain activity differences and similarities between major depression (MDD) and bipolar disorder (BD) is necessary. However, results have not yet yielded consistent conclusions. A meta-analysis of whole-brain resting-state functional MRI (rs-fMRI) studies that explored differences in the amplitude of low-frequency fluctuation (ALFF) between patients (including MDD and BD) and healthy controls (HCs) was conducted using seed-based d mapping software. Systematic literature search identified 50 studies comparing 1399 MDD patients and 1332 HCs, and 15 studies comparing 494 BD patients and 593 HCs. MDD patients displayed increased ALFF in the right superior frontal gyrus (SFG) (including the medial orbitofrontal cortex, medial prefrontal cortex [mPFC], anterior cingulate cortex [ACC]), bilateral insula extending into the striatum and left supramarginal gyrus and decreased ALFF in the bilateral cerebellum, bilateral precuneus, and left occipital cortex compared with HCs. BD showed increased ALFF in the bilateral inferior frontal gyrus, bilateral insula extending into the striatum, right SFG, and right superior temporal gyrus (STG) and decreased ALFF in the bilateral precuneus, left cerebellum (extending to the occipital cortex), left ACC, and left STG. In addition, MDD displayed increased ALFF in the left lingual gyrus, left ACC, bilateral precuneus/posterior cingulate gyrus, and left STG and decreased ALFF in the right insula, right mPFC, right fusiform gyrus, and bilateral striatum relative to BD patients. Conjunction analysis showed increased ALFF in the bilateral insula, mPFC, and decreased ALFF in the left cerebellum in both disorders. Our comprehensive meta-analysis suggests that MDD and BD show a common pattern of aberrant regional intrinsic brain activity which predominantly includes the insula, mPFC, and cerebellum, while the limbic system and occipital cortex may be associated with spatially distinct patterns of brain function, which provide useful insights for understanding the underlying pathophysiology of brain dysfunction in affective disorders, and developing more targeted and efficacious treatment and intervention strategies.