One hallmark of Alzheimer´s disease are senile plaques consisting of amyloid beta (Aβ), which derives from the processing of the amyloid precursor protein (APP). Mitochondrial dysfunction has been linked to the pathogenesis of Alzheimer´s disease and both Aβ and APP have been reported to affect mitochondrial function in isolated systems. However, in intact cells, considering a physiological localization of APP and Aβ, it is pending what triggers the mitochondrial defect. Thus, the aim of this study was to dissect the impact of APP versus Aβ in inducing mitochondrial alterations with respect to their subcellular localization. We performed an overexpression of APP or beta-site amyloid precursor protein cleaving enzyme 1 (BACE1), increasing APP and Aβ levels or Aβ alone, respectively. Conducting a comprehensive metabolic characterization we demonstrate that only APP overexpression reduced mitochondrial respiration, despite lower extracellular Aβ levels compared to BACE overexpression. Surprisingly, this could be rescued by a gamma secretase inhibitor, oppositionally indicating an Aβ-mediated mitochondrial toxicity. Analyzing Aβ localization revealed that intracellular levels of Aβ and an increased spatial association of APP/Aβ with mitochondria are associated with reduced mitochondrial respiration. Thus, our data provide marked evidence for a prominent role of intracellular Aβ accumulation in Alzheimer´s disease associated mitochondrial dysfunction. Thereby it highlights the importance of the localization of APP processing and intracellular transport as a decisive factor for mitochondrial function, linking two prominent hallmarks of neurodegenerative diseases.