We have explored the hypothesis that hypertrophy of vascular smooth muscle cells may be regulated, in part, by growth inhibitory factors that alter the pattern of the growth response to serum mitogens by characterizing the effects of the potent growth inhibitor, transforming growth factor-beta (TGF-beta), on both hyperplastic and hypertrophic growth of cultured rat aortic smooth muscle cells. TGF-beta inhibited serum-induced proliferation of rat aortic smooth muscle cells (ED50 = 2 pM); this is consistent with previously reported observations in bovine aortic smooth muscle cells (Assoian et al. 1982. J. Biol. Chem. 258:7155-7160). Growth inhibition was due in part to a greater than twofold increase in the cell cycle transit time in cells that continued to proliferate in the presence of TGF-beta. TGF-beta concurrently induced cellular hypertrophy as assessed by flow cytometric analysis of cellular protein content (47% increase) and forward angle light scatter (32-50% increase), an index of cell size. In addition to being time and concentration dependent, this hypertrophy was reversible. Simultaneous flow cytometric evaluation of forward angle light scatter and cellular DNA content demonstrated that TGF-beta-induced hypertrophy was not dependent on withdrawal of cells from the cell cycle nor was it dependent on growth arrest of cells at a particular point in the cell cycle in that both cycling cells in the G2 phase of the cell cycle and those in G1 were hypertrophied with respect to the corresponding cells in vehicle-treated controls. Chronic treatment with TGF-beta (100 pM, 9 d) was associated with accumulation of cells in the G2 phase of the cell cycle in the virtual absence of cells in S phase, whereas subsequent removal of TGF-beta from these cultures was associated with the appearance of a significant fraction of cycling cells with greater than 4c DNA content, consistent with development of tetraploidy. Results of these studies support a role for TGF-beta in the control of smooth muscle cell growth and suggest that at least one mechanism whereby hypertrophy and hyperploidy may occur in this, as well as other cell types, is by alterations in the response to serum mitogens by potent growth inhibitors such as TGF-beta.