Understanding dose-dependent survival of irradiated cells is a pivotal goal in radiotherapy and radiobiology. To this end, the clonogenic assay is the standard in vitro method, classifying colonies into either clonogenic or non-clonogenic based on a size threshold at a fixed time. Here we developed a methodological framework for the automated analysis of time course live-cell image data to examine in detail the growth dynamics of large numbers of colonies that occur during such an experiment. We developed a segmentation procedure that exploits the characteristic composition of phase-contrast images to identify individual colonies. Colony tracking allowed us to characterize colony growth dynamics as a function of dose by extracting essential information: (a) colony size distributions across time; (b) fractions of differential growth behavior; and (c) distributions of colony growth rates across all tested doses. We analyzed three data sets from two cell lines (H3122 and RENCA) and made consistent observations in line with already published results: (i) colony growth rates are normally distributed with a large variance; (ii) with increasing dose, the fraction of exponentially growing colonies decreases, whereas the fraction of delayed abortive colonies increases; as a novel finding, we observed that (iii) mean exponential growth rates decrease linearly with increasing dose across the tested range (0-10 Gy). The presented method is a powerful tool to examine live colony growth on a large scale and will help to deepen our understanding of the dynamic, stochastic processes underlying the radiation response in vitro.