This paper addresses the state estimation of the nonlinear initial alignment of the strapdown inertial navigation system (SINS), which mainly focuses on the initial alignment on the swaying base and under the in-motion condition with the measurement uncertainties. In order to achieve a higher alignment precision, stronger numerical stability, and lower computational cost for the initial alignment of SINS on the swaying base, a new discrete large azimuth misalignment error model of SINS is established, and an improved fifth-degree cubature Kalman filter (5th-CKF) algorithm is proposed, which combines the 5th-CKF and a simplified dimensionality reduction filtering algorithm. The 5th-CKF is introduced to solve the nonlinear filtering problem, a simplified dimensionality reduction algorithm is derived to reduce the large calculation values of 5th-CKF. Furthermore, under the Bayesian framework, a novel filtering approach named the fifth-degree variational Bayesian (VB) adaptive cubature Kalman filter is deduced for the in-motion alignment with a large azimuth misalignment angle and unknown and time-varying measurement noise statistics, which combines the iterative VB approach and 5th-CKF. The 5th-CKF is exploited to handle the nonlinear initial alignment model, and the VB approach is utilized to iteratively estimate the sufficient statistics of the measurement noise. Mathematical simulation, turntable, and vehicle experiments are performed to demonstrate the effectiveness and the superiority of the proposed approaches.