Endoscopy serves as an indispensable minimally-invasive surgical procedure. Due to the limited view and non-intuitive operation of the instrument, the mastery of endoscopic manipulation requires deep medical knowledge as well as complex perception and motor skills of the surgeon. Intensive surgical training is required, and simulation-based training is of more and more importance over traditional animal- or cadaver-based approaches. Here, we developed a hybrid surgical simulator that consists of a realistic physical organ model and an artificial intelligence (AI)-driven cyber model. We built a physical model of the full urinary tract with soft materials and detailed blood vessel structures. Endourological procedures were performed to localize and treat renal calculi by a flexible endoscope. An AI algorithm detects the lesions automatically with high accuracy and provides quantitative feedback about an operator's endoscopic skills. The hybrid simulator system shows great potential as an interactive and personalized learning environment for endoscopic skills. Clinical Relevance- This work establishes a preliminary approach for realistic endoscopic training. The developed hybrid surgical simulator - with high-fidelity physical organ models and quantitative feedback - can deliver effective hands-on learning to surgeons to improve their endoscopic skills.