Gastric cancer is a highly prevalent cancer world-wide. Accurate diagnosis of Early Gastric Cancer (EGC) is of great significance to improve the treatment and survival rate of patients. However, EGC and gastric ulcers have similar en-doscopic image characteristics, resulting in a high misdiagnosis rate. Most existing deep learning and machine learning models for EGC recognition have the disadvantages of cumbersome pre-processing steps and high leakage ratios. To address the above challenges, we propose an end-to-end Adversarial Do-main Adaptation Neural network (ADAN) for EGC prediction on endoscopic images. ADAN network consists of a source domain feature extractor, a source domain classifier, two target domain feature extractors, a target domain classifier, and a domain discriminator. A source domain feature extractor is designed to train the model on public gastrointestinal datasets, which effectively solves the problem of insufficient training data. In addition, an adaptive source-target domain mapping classifier is added to each target domain feature extractor for automatically adjusting the number of classification categories in the target domain. Experimental results show that the proposed ADAN network is superior to the most advanced methods and can accurately predict EGC in clinical practice. Clinical relevance-In this study, the EGC diagnosis model based on the adversarial domain adaptive construction will be more applicable to the real clinical scenario, with higher accuracy and sensitivity and assist the endoscopist to make more accurate diagnosis for EGC and reduce the workload.