Isolated mitochondria may undergo uncoupling, and in presence of Ca(2+) at different conditions, a mitochondrial permeability transition (MPT) linked to protein thiol oxidation, and demonstrated by CsA-sensitive mitochondrial swelling; these processes may cause cell death either by necrosis or by apoptosis. Isocoumarins isolated from the Brazilian plant Paepalanthus bromelioides (Eriocaulaceae) paepalantine (9,10-dihydroxy-5,7-dimethoxy-1H-naptho(2,3c)pyran-1-one), 8,8'-paepalantine dimer, and vioxanthin were assayed at 1-50 microM on isolated rat liver mitochondria, for respiration, MPT, protein thiol oxidation, and interaction with the mitochondrial membrane using 1,6-diphenyl-1,3,5-hexatriene (DPH). The isocoumarins did not significantly affect state 3 respiration of succinate-energized mitochondria; they did however, stimulate 4 respiration, indicating mitochondrial uncoupling. Induction of MPT and protein thiol oxidation were assessed in succinate-energized mitochondria exposed to 10 microM Ca(2+); inhibition of these processes was assessed in non-energized organelles in the presence of 300 microM t-butyl hydroperoxide plus 500 microM Ca(2+). Only paepalantine was an effective MPT/protein thiol oxidation inducer, also releasing cytochrome c from mitochondria; the protein thiol oxidation, unlike mitochondrial swelling, was neither inhibited by CsA nor dependent on the presence of Ca(2+). Vioxanthin was an effective inhibitor of MPT/protein thiol oxidation. All isocoumarins inserted deeply into the mitochondrial membrane, but only paepalantine dimer and vioxantin decreased the membrane's fluidity. A direct reaction with mitochondrial membrane protein thiols, involving an oxidation of these groups, is proposed to account for MPT induction by paepalantine, while a restriction of oxidation of these same thiol groups imposed by the decrease of membrane fluidity, is proposed to account for MPT inhibition by vioxanthin.