We have previously shown that transforming growth factor beta 1 (TGF-beta 1) receptor and TGF-beta type II receptor (T beta R-II) are produced in lactotropes, and that TGF-beta 1 inhibits the growth of these anterior pituitary cells by an autocrine mechanism. To study the changes of the expression and function of this growth factor during tumorigenesis, we have measured the levels of TGF-beta 1 and T beta R-II mRNAs and proteins in the normal and tumor anterior pituitary cells in vivo and in vitro and have compared the cell growth responses to TGF-beta 1 in normal and tumor pituitary cells in vitro. Treatment with estradiol-17 beta for 1, 2, 4, and 8 weeks caused a time-dependent increase in pituitary protein, prolactin, and prolactin mRNA levels and in plasma prolactin levels, suggesting that estrogen enhanced lactotropic proliferation in anterior pituitary glands. The levels of TGF-beta 1 protein and mRNA in anterior pituitary tissues were reduced over time after estrogen treatment during the development of pituitary tumors. The mRNA and protein levels of T beta R-II decreased markedly during the development of pituitary tumors. In addition, two transformed lactotropes, GH3 and PR1 cell lines, showed markedly reduced levels of TGF-beta 1 as well as T beta R-II mRNA. Comparison of the antiproliferative effects of TGF-beta in transformed and normal lactotropes in cultures revealed that the sensitivity of GH3 cells is reduced, and that PR1 cells are virtually resistant to TGF-beta 1.(ABSTRACT TRUNCATED AT 250 WORDS)