Breast cancer is a complex disease with many distinct subtypes being recognized on the basis of histological features and molecular signatures. It is difficult to predict how cancers will respond to therapy, which results in many women receiving unnecessary or inappropriate treatment. Advances in materials science and tissue engineering are leading the development of complex in vitro 3D breast tissue models that will increase our understanding of normal development and tumorigenic mechanisms. Ultimately, platforms that support primary tissue culture could readily be adapted to form high-throughput drug screening tools for personalized medicine. This review will summarize the control of mammary gland phenotype within in vitro 3D environments, in the context of a detailed analysis of mammary gland development and stem and progenitor cell controlled tumorigenesis.