Predicting tumor growth and its response to therapy remains a major challenge in cancer research and strongly relies on tumor growth models. In this paper, we introduce, calibrate, and verify a novel image-driven reaction-diffusion model of avascular tumor growth. The model allows for proliferation, death and spread of tumor cells, and accounts for nutrient distribution and hypoxia. It is constrained by longitudinal time series of dynamic contrast-enhancement-MRI images. Tumor specific parameters are estimated from two early time points and used to predict the spatio-temporal evolution of the tumor volume and cell densities at later time points. We first test our parameter estimation approach on synthetic data from 15 generated tumors. Our in silico study resulted in small volume errors (<5%) and high Dice overlaps (>97%), showing that model parameters can be successfully recovered and used to accurately predict the tumor growth. Encouraged by these results, we apply our model to seven pre-clinical cases of breast carcinoma. We are able to show promising preliminary results, especially for the estimation for early time points. Processes like angiogenesis and apoptosis should be included to further improve predictions for later time points.