In this paper, a hybrid model is proposed to simultaneously mine causal connections and features responsible for contemporaneous correlations in a multivariate process. The model is developed by combining the vector auto-regressive exogenous model and the factor analysis model. The parameters of the resulting model are regularized using the hierarchical prior distributions for pruning insignificant/irrelevant ones from the model. It is then estimated under the variational Bayesian expectation maximization framework. The estimation is initiated with a complex model which is then systematically reduced to a simpler model that retains only the parameters corresponding to significant causal connections and contemporaneous correlations. Model reduction is carried out through a series of deterministic jumps from complex models to simpler models using a relevance criterion. The approach is illustrated with a number of simulated examples and an industrial case study.