Nanoparticle (NP)-mediated drug delivery (NMDD) has emerged as a novel method to overcome the limitations of traditional systemic delivery of therapeutics, including the controlled release of the NP-associated drug cargo. Currently, our most advanced understanding of how to control NP-associated cargos is in the context of soft nanoparticles (e.g., liposomes), but less is known about controlling the release of cargos from the surface of hard NPs (e.g., gold NPs). Here we employ a semiconductor quantum dot (QD) as a prototypical hard NP platform and use intracellularly triggered actuation to achieve spatiotemporal control of drug release and modulation of drug efficacy. Conjugated to the QD are two peptides: (1) a cell-penetrating peptide (CPP) that facilitates uptake of the conjugate into the endocytic pathway and (2) a display peptide conjugated to doxorubicin (DOX) via three different linkages (ester, disulfide, and hydrazone) that are responsive to enzymatic cleavage, reducing conditions, and low pH, respectively. Formation of the QD-[peptide-DOX]-CPP complex is driven by self-assembly that allows control over both the ratio of each peptide species conjugated to the QD and the eventual drug dose delivered to cells. Förster resonance energy transfer assays confirmed successful assembly of the QD-peptide complexes and functionality of the linkages. Confocal microscopy was employed to visualize residence of the QD-[peptide-DOX]-CPP complexes in the endocytic pathway, and distinct differences in DOX localization were noted for the ester linkage, which showed clear signs of nuclear delivery versus the hydrazone, disulfide, and amide control. Finally, delivery of the QD-[peptide-DOX]-CPP conjugate resulted in cytotoxicity for the ester linkage that was comparable to free DOX. Attachment of DOX via the hydrazone linkage facilitated intermediary toxicity, while the disulfide and amide control linkages showed minimal toxicity. Our data demonstrate the utility of hard NP-peptide bioconjugates to function as multifunctional scaffolds for simultaneous control over cellular drug uptake and toxicity and the vital role played by the nature of the chemical linkage that appends the drug to the NP carrier.