A 3D+t description of the coronary tree is important for diagnosis of coronary artery disease and therapy planning. In this paper, we propose a method for finding 3D+t points on coronary artery tree given tracked 2D+t point locations in X-ray rotational angiography images. In order to cope with the ill-posedness of the problem, we use a bilinear model of ventricle as a spatio-temporal constraint on the nonrigid structure of the coronary artery. Based on an energy minimization formulation, we estimate i) bilinear model parameters, ii) global rigid transformation between model and X-ray coordinate systems, and iii) correspondences between 2D coronary artery points on X-ray images and 3D points on bilinear model. We validated the algorithm using a software coronary artery phantom.