The ray transfer matrix method is widely used for analyzing beam-transmission properties and designing multi-pass systems. In this paper, a modified ray transfer matrix method is proposed for tracing an accurate non-sequential ray in three-dimensional space based on a vector reflection theory. The modified ray transfer matrix method can be utilized to analyze a ray transmitting between arbitrary surfaces, which is not confined to rotational symmetrical structures. There is no need to project rays onto two perpendicular planes, nor to introduce a paraxial approximation in our calculation. Since the length and angle of every ray can be calculated accurately, almost no deviation is accumulated in multi-pass system. The modified ray transfer matrix method can be applied in optical design, especially in the design of multi-pass cavities and multi-pass cells.