Ternary Cu(II) complexes with bidentate malonato- and heterocyclic amine ligands were tested with regard to cytotoxicity and potentiation of x-ray induced cell killing in V79 cells. Two lead complexes were also tested in a tumor assay using the MTG-B murine adenocarcinoma model growing in the flanks of female C3H/HeJ mice. One complex, [2,2'-bipyridyl malonatoCu(II)] (RL-5077), produced sensitizer enhancement ratios (SER's) of 1.8 (hypoxic conditions) and 1.0 (oxic conditions) in vitro when irradiation followed 1 hr exposure to the drug at 100 microM. When RL-5077 was administered at doses of 1/2 (11.65 mg/kg) or 1/4 (5.25 mg/kg) the maximum tolerated dose (MTD), 15 min prior to a locally delivered dose of 20 Gy, enhancement ratios (ER's) of 1.6 and 2, respectively, resulted. The second lead complex, [1,10 phenanthroline (malonato)Cu(II)hydrate] (RL-5027), produced SER's of 1.8 and 1.2 under hypoxic and oxic conditions, respectively, at a concentration of 25 microM. Injection of RL-5027 (5 mg/kg) resulted in toxicity without enhancement in combination with radiation. Analogues of these two complexes have been synthesized in an effort to optimize the potentiation of radiation effects while minimizing toxicity to drug alone and increasing water solubility of the drug. Further studies of the structure-activity relationship of Cu(II) ternary complexes using in vitro radiosensitization as the endpoint have identified four classes of ligands with varying biological activity and have supplied information about the effects of group substitution on solubility, toxicity, and radiation potentiation. This group of complexes represents a new class of radiopotentiators that deserves further investigation into its potential for clinical use.