Scoliosis is a common spinal condition where the spine curves to the side and thus deforms the spine. Curvature estimation provides a powerful index to evaluate the deformation severity of scoliosis. In current clinical diagnosis, the standard curvature estimation method for assessing the curvature quantitatively is done by measuring the Cobb angle, which is the angle between two lines, drawn perpendicular to the upper endplate of the uppermost vertebra involved and the lower endplate of the lowest vertebra involved. However, manual measurement of spine curvature requires considerable time and effort, along with associated problems such as interobserver and intraobserver variations. In this article, we propose an automatic system for measuring spine curvature using the anterior-posterior (AP) view spinal X-ray images. Due to the characteristic of AP view images, we first reduced the image size and then used horizontal and vertical intensity projection histograms to define the region of interest of the spine which is then cropped for sequential processing. Next, the boundaries of the spine, the central spinal curve line, and the spine foreground are detected by using intensity and gradient information of the region of interest, and a progressive thresholding approach is then employed to detect the locations of the vertebrae. In order to reduce the influences of inconsistent intensity distribution of vertebrae in the spine AP image, we applied the deep learning convolutional neural network (CNN) approaches which include the U-Net, the Dense U-Net, and Residual U-Net, to segment the vertebrae. Finally, the segmentation results of the vertebrae are reconstructed into a complete segmented spine image, and the spine curvature is calculated based on the Cobb angle criterion. In the experiments, we showed the results for spine segmentation and spine curvature; the results were then compared to manual measurements by specialists. The segmentation results of the Residual U-Net were superior to the other two convolutional neural networks. The one-way ANOVA test also demonstrated that the three measurements including the manual records of two different physicians and our proposed measured record were not significantly different in terms of spine curvature measurement. Looking forward, the proposed system can be applied in clinical diagnosis to assist doctors for a better understanding of scoliosis severity and for clinical treatments.