Pancreaticobiliary carcinomas is a highly malignant gastrointestinal tumor. Most pancreaticobiliary cancers arise from epithelial proliferation within the pancreaticobiliary ducts, referred to as pancreatic intraepithelial neoplasias (PanINs). Some PanINs are benign metaplasia, while others progress to invasive duct adenocarcinoma (IDAC). However, there is no standard program to diagnose the progression from PanINs to IDAC. In this study, we present a tri-modality imaging system, which integrates ultrasound (US), optical coherence tomography (OCT), and near-infrared fluorescence (NIRF) for pancreaticobiliary duct imaging. This system can obtain OCT, US, and NIRF images in real-time with a frame rate of 30 frames per second. For the endoscopy probe with an outer diameter of 0.9 mm, the US transducer and fiber ball lens were placed back to back. In vivo experiments were performed on the rectums of Sprague-Dawley rats to demonstrate the imaging performance of US, OCT, and fluorescence angiography. An ex vivo experiment on a human pancreatic duct was performed for a more accurate assessment of the pancreaticobiliary duct. The tomography images of rat rectums and human pancreatic ducts were correlated with hematoxylin and eosin (H&E) histology to check the measurement accuracy. The integrated tri-modality system has great clinical potential in mechanism studies, early diagnosis, and prognosis evaluation of malignant pancreaticobiliary carcinomas.