The hollow fiber assay, a unique in vivo model, permits the simultaneous evaluation of compound efficacy against multiple cell lines in two physiological compartments. This assay has been used to characterize in vivo activity of cytotoxic compounds. The purpose of the present study was to characterize and optimize this assay for compounds with a defined mechanism of action, specifically cell cycle inhibition. Two human tumor cell lines and one normal human cell line were loaded into polyvinylidene fluoride hollow fibers at two or more cell concentrations and grown in mice for 3-10 days. The data demonstrate the importance of characterizing the initial loading density of various cell lines in the evaluation of compounds. All studies were performed with cells in the linear part of the cell growth curves. Initial loading densities of 1-2 x 10(4) cells/fiber gave the greatest opportunity for growth in the three human cell lines tested (HCT116 colon carcinoma, NCI-H460 non-small cell carcinoma, and AG 1523 normal fibroblast). Utilizing the MTT assay, standard curves were constructed to correlate the final number of cells with optical density (OD) readings at 540 nm in order to calculate cell numbers in the fibers. Insights into the mechanism of action of cisplatin have been gained using Western blot analysis of the cell cycle markers PCNA (a protein present throughout the cell cycle) and Rb (a protein that acts as a tumor suppressor gene product) from the hollow fiber cells. In cisplatin-treated NCI-H460 cells both PCNA and Rb phosphorylation decreased, suggesting the arrest of the cells prior to the S phase. Standard therapeutic agents, cisplatin, racemic flavopiridol, cyclophosphamide and mitomycin C, were evaluated independently in the hollow fiber assay and the xenograft model. The data demonstrate that compounds active in the hollow fiber assay are also active in the xenograft.