Fractal analyses have been applied successfully for the image compression, texture analysis, and texture image segmentation. The fractal dimension could be used to quantify the texture information. In this study, the differences of gray value of neighboring pixels are used to estimate the fractal dimension of an ultrasound image of breast lesion by using the fractal Brownian motion. Furthermore, a computer-aided diagnosis (CAD) system based on the fractal analysis is proposed to classify the breast lesions into two classes: benign and malignant. To improve the classification performances, the ultrasound images are preprocessed by using morphology operations and histogram equalization. Finally, the k-means classification method is used to classify benign tumors from malignant ones. The US breast image databases include only histologically confirmed cases: 110 malignant and 140 benign tumors, which were recorded. All the digital images were obtained prior to biopsy using by an ATL HDI 3000 system. The receiver operator characteristic (ROC) area index AZ is 0.9218, which represents the diagnostic performance.