We present a system for monitoring patient respiration using a single low cost camera. The system is based on tracking the motion of color fiducials placed on the patients abdomen. We show that our system is able to deal with variations in camera pose, and that its performance is equivalent to that obtained when using a commercial tracking system. We also validate our system using ultrasound images showing that it is sufficiently accurate for gating purposes in radiation treatment or for acquiring 4D (3D over time) data sets.