Recently, much attention has been focused on heart motion analysis for minimally invasive beating-heart surgery. Unfortunately existing techniques usually require the camera(s) to be fixed during the motion analysis, which can restrict its usefulness during surgery. In this paper we present a novel method for heart motion analysis using geometric constraint, which can estimate the motion from a moving camera and employ multiple image features to improve robustness to noise. Our approach combines the benefits of geometry estimation for obtaining an accurate and robust solution with the proper treatment of respiratory motion. The proposed method can be applied to not only beating heart surgery, but also to other procedures involving periodic organ motion, such as lung and liver.