The accuracy of an ultrasound (US) computer-aided diagnosis (CAD) system was evaluated for the classification of BI-RADS category 3, probably benign masses. The US database used in this study contained 69 breast masses (21 malignant and 48 benign masses) that at blinded retrospective interpretation were assigned to BI-RADS category 3 by at least one of five radiologists. For computer-aided analysis, multiple morphology (shape, orientation, margin, lesions boundary, and posterior acoustic features) and texture (echo patterns) features based on BI-RADS lexicon were implemented, and the binary logistic regression model was used for classification. The receiver operating characteristic curve analysis was used for statistical analysis. The area under the curve (Az) of morphology, texture, and combined features were 0.90, 0.75, and 0.95, respectively. The combined features achieved the best performance and were significantly better than using texture features only (0.95 vs. 0.75, p value = 0.0163). The cut-off point at the sensitivity of 86 % (18/21), 95 % (20/21), and 100 % (21/21) achieved the specificity of 90 % (43/48), 73 % (35/48), and 33 % (16/48), respectively. In conclusion, the proposed CAD system has the potential to be used in upgrading malignant masses misclassified as BI-RADS category 3 on US by the radiologists.