An accurate identification and localization of vertebrae in X-ray images can assist doctors in measuring Cobb angles for treating patients with adolescent idiopathic scoliosis. It is useful for clinical decision support systems for diagnosis, surgery planning, and spinal health analysis. Currently, publicly available annotated datasets on spinal vertebrae are small, making deep-learning-based detection methods that are highly data-dependent less accurate. In this article, we propose an algorithm based on convolutional neural networks that can be trained to detect vertebrae from a small set of images. This method can display critical information on a patient's spine, display vertebrae and their labels on the thoracic and lumbar, calculate the Cobb angle, and evaluate the severity of spinal deformities. The proposed achieved an average accuracy of 0.958 and 0.962 for classifying spinal deformities (i.e., C-shaped, S-shaped type 1, and S-shaped type 2) and severity of Cobb angle (i.e., normal, mild, moderate, and severe), respectively. The Cobb angle measurement had a median difference of less than 5° from the ground-truth with SMAPE of 5.27% and an error on landmark detection of 19.73. In addition, Lenke classification is used to analyze spinal deformities as types A, B, and C, which have an average accuracy of 0.924. Physicians can use the proposed system in clinical practice by providing X-ray images via the user interface.