This article proposes a robust Bayesian inference approach for linear state-space models with nonstationary and heavy-tailed noise for robust state estimation. The predicted distribution is modeled as the hierarchical Student- t distribution, while the likelihood function is modified to the Student- t mixture distribution. By learning the corresponding parameters online, informative components of the Student- t mixture distribution are adapted to approximate the statistics of potential uncertainties. Then, the obstacle caused by the coupling of the updated parameters is eliminated by the variational Bayesian (VB) technique and fixed-point iterations. Discussions are provided to show the reasons for the achieved advantages analytically. Using the Newtonian tracking example and a three degree-of-freedom (DOF) hover system, we show that the proposed inference approach exhibits better performance compared with the existing method in the presence of modeling uncertainties and measurement outliers.