A regularized color clustering algorithm is proposed to solve the color clustering problem in medical image database. By incorporating both measures of cluster separability and cluster compactness, regularized color clustering allows the automatic extraction of significant color groups with varying populations. Experimental results in different color spaces show that the regularized color clustering gives superior results in extracting significant distinct/abnormal color clusters without significant increases in cluster compactness. Furthermore, results of color clustering in different color spaces show that the LUV color space is more suitable for color clustering. Methods for selecting the regularization constants have also been suggested.