This paper presents a novel self-calibration method of an X-ray scene applied for the 3-D reconstruction of the scoliotic spine. Current calibration techniques either use a cumbersome calibration apparatus or depend on manually identified landmarks to determine the geometric configuration, thus limiting routine clinical evaluation. The proposed approach uses high-level information automatically extracted from biplanar X-rays to solve the radiographic scene parameters. We first present a segmentation method that takes into account the variable appearance and geometry of a scoliotic spine in order to isolate and extract the silhouettes of the anterior vertebral body. By incorporating prior anatomical information through a Bayesian formulation of the morphological distribution, a multiscale spine segmentation framework is proposed for scoliotic patients. An iterative nonlinear optimization procedure, integrating a 3-D visual hull reconstruction and geometrical torsion properties of the spine, is then applied to globally refine the geometrical parameters of the 3-D viewing scene and obtain the optimal 3-D reconstruction. An experimental comparison with data provided from reference synthetic models yields similar accuracy on the retroprojection of low-level primitives such as anatomical landmarks identified on each vertebra (2.2 mm). Results obtained from a clinical validation on 60 pairs of uncalibrated digitized X-rays of adolescents with scoliosis show that the 3-D reconstructions from the new system offer geometrically accurate models with insignificant differences for 3-D clinical indexes commonly used in the evaluation of spinal deformities. The reported experiments demonstrate a viable and accurate alternative to previous reconstruction techniques, offering the first automatic approach for routine 3-D clinical assessment in radiographic suites.