Tracking instruments during surgery is becoming a useful acquisition tool for different applications. This article presents a tracking system to detect and track instruments in endoscopic video using biocompatible colour markers. The system tracks single or multiple instruments in the video. The originality of this method is that it combines continuously adaptive shift algorithm with Kalman-filter for real-time tracking of single and multiple surgical instruments during surgery. Preliminary results show that the proposed method has a real-time performance. Moreover it is robust to partial occlusion and smoke. The system shows high sensitivity and specificity results for blue, green and yellow colours. The achieved sensitivity and specificity results are sufficient to apply the system for real-time registration of surgical workflow in-vivo during surgery.