Targeted delivery combined with controlled drug release has a pivotal role in the future of personalized medicine. This review covers the principles, advantages, and drawbacks of passive and active targeting based on various polymer and magnetic iron oxide nanoparticle carriers with drug attached by both covalent and noncovalent pathways. Attention is devoted to the tailored conjugation of targeting ligands (e.g., enzymes, antibodies, peptides) to drug carrier systems. Similarly, the approaches toward controlled drug release are discussed. Various polymer-drug conjugates based, for example, on polyethylene glycol (PEG), N-(2-hydroxypropyl)methacrylamide (HPMA), polymeric micelles, and nanoparticle carriers are explored with respect to absorption, distribution, metabolism, and excretion (ADME scheme) of administrated drug. Design and structure of superparamagnetic iron oxide nanoparticles (SPION) and condensed magnetic clusters are classified according to the mechanism of noncovalent drug loading involving hydrophobic and electrostatic interactions, coordination chemistry, and encapsulation in porous materials. Principles of covalent conjugation of drugs with SPIONs including thermo- and pH-degradable bonds, amide linkage, redox-cleavable bonds, and enzymatically-cleavable bonds are also thoroughly described. Finally, results of clinical trials obtained with polymeric and magnetic carriers are analyzed highlighting the potential advantages and future directions in targeted anticancer therapy.