One of the most challenging problems in robot-assisted surgical systems is to provide surgical realism at interactive simulation rates. The proposed visual tracking system can track and register object deformations in real time using a physically based formulation, despite the occlusions produced by the robotic system itself. The results obtained provide an accurate visual representation of the deformed solid and will thus enable new assistance approaches to help surgeons during surgical procedures.