To continue elucidation of the biochemical and molecular pathways involved in the induction of apoptosis in granulosa cells (GC) of ovarian follicles destined for atresia, we characterized the occurrence and protease modulation of high and low molecular weight (MW) DNA fragmentation during rat GC death. Atresia of ovarian follicles, occurring either spontaneously in vivo or induced in vitro, was associated with both high MW and internucleosomal (low MW) DNA cleavage. Incubation of follicles in the presence of a putative irreversible and non-competitive inhibitor of caspase-1 (interleukin-1beta-converting enzyme or ICE), sodium aurothiomalate (SAM), completely prevented internucleosomal, but not high MW, DNA cleavage. As reported previously, morphological features of apoptosis (pyknosis, cellular condensation) and atresia (granulosa cell disorganization, oocyte pseudomaturation) remained detectable in SAM-treated follicles. The potential involvement of proteases in endonuclease activation was further analyzed in cell-free assays using nuclei from both GC (which autodigest their DNA) and HeLa cells (HC, which do not autodigest their DNA unless incubated with extracts prepared from other cell types). Crude cytoplasmic extracts prepared from GC induced both high MW and internucleosomal DNA cleavage in HC nuclei. The induction of low, but not high, MW DNA cleavage in HC nuclei by GC extracts was suppressed by pretreatment of the extracts with SAM or with any one of the serine protease inhibitors, dichloroisocoumarin (DCI), N-tosyl-L-leucylchloromethylketone (TLCK) or N-tosyl-L-phenylchloromethylketone (TPCK). Interestingly, SAM and DCI also prevented cation-induced low MW DNA fragmentation in GC nuclei; however, TLCK and TPCK were without effect. Our results support a role for cytoplasmic and nuclear serine proteases in the activation of the endonuclease(s) responsible for internucleosomal DNA cleavage during apoptosis.