The American Cancer Society (ACS) recommends women aged 40 and above to have a mammogram every year and calls it a gold standard for breast cancer detection. Early detection of breast cancer can improve survival rates to a great extent. Inter-observer and intra-observer errors occur frequently in analysis of medical images, given the high variability between interpretations of different radiologists. Also, the sensitivity of mammographic screening varies with image quality and expertise of the radiologist. So, there is no golden standard for the screening process. To offset this variability and to standardize the diagnostic procedures, efforts are being made to develop automated techniques for diagnosis and grading of breast cancer images. A few papers have documented the general trend of computer-aided diagnosis of breast cancer, making a broad study of the several techniques involved. But, there is no definitive documentation focusing on the mathematical techniques used in breast cancer detection. This review aims at providing an overview about recent advances and developments in the field of Computer-Aided Diagnosis (CAD) of breast cancer using mammograms, specifically focusing on the mathematical aspects of the same, aiming to act as a mathematical primer for intermediates and experts in the field.