Transforming growth factor beta (TGF) is a well-known inhibitor of myogenic differentiation as well as an autocrine product of rhabdomyosarcoma cells. We studied the role of the TGF-beta autocrine loop in regulating growth and myogenic differentiation in the human rhabdomyosarcoma cell line, RD. We previously reported that the phorbol ester 12-O-tetradecanoylphorbol-13-acetate (TPA) induces growth arrest and myogenic differentiation in these cells, which constitutively express muscle regulatory factors. We show that TPA inhibits the activation of secreted latent TGF-beta, thus decreasing the concentration of active TGF-beta to which the cells are exposed. This event is mediated by the TPA-induced alteration of the uPA/PAI serine-protease system. Complete removal of TGF-beta, mediated by the ectopic expression of a soluble type II TGF-beta receptor dominant negative cDNA, induces growth arrest, but does not trigger differentiation. In contrast, a reduction in the TGF-beta concentration, to a range of 0.14-0.20 x 10(-2) ng/ml (which is similar to that measured in TPA-treated cells), mimics TPA-induced differentiation. Taken together, these data demonstrate that cell growth and suppression of differentiation in rhabdomyosarcoma cells require overproduction of active TGF-beta; furthermore, they show that a 'critical' concentration of TGF-beta is necessary for myogenic differentiation to occur, whereas myogenesis is abolished below and above this concentration. By impairing the TGF-beta autocrine loop, TPA stabilizes the factor concentration within the range compatible for differentiation to occur. In contrast, in human primary muscle cells a much higher concentration of exogenous TGF-beta is required for the differentiation inhibitory effect and TPA inhibits differentiation in these cells probably through a TGF-beta independent mechanism. These data thus clarify the mechanism underlying the multiple roles of TGF-beta in the regulation of both the transformed and differentiated phenotype.