The aim of this study was to scrutinize the evidence of adjuvant chemotherapy of gastric cancer by assessing the quality of the medical literature of randomized controlled trials (RCTs). A quality assessment (QA) scoring system was devised with the three parameters-control of bias, quality of report, and quality of design-which consisted 19 items. We searched for all the publications of the RCTs, from 1969 to 2007, with surgery-only arm, and their associated meta-analyses to score. Among the 26 RCTs, quality of three articles were graded as (2+), 10 articles as (1+), and 13 articles as (-). Recently published studies had overall better quality of report, but not necessarily better quality of design. Three studies demonstrating a positive survival benefit of adjuvant chemotherapy had a grade (1+). Hierarchical clustering revealed that the 26 articles were grouped into three major branches associated with study quality and a multi-institutional setting. We also obtained a statistically significant set of ten items (P < 0.001) that could differentiate articles of good (1-2+) and low quality (-) through supervised two-way hierarchical clustering. Finally, the level of recommendation for adjuvant chemotherapy in gastric cancer was to be a "B" according to the Scottish Intercollegiate Guidelines Network (SIGN) System. QA of medical literature should be an essential consideration for medical-related decision-making and the formation of evidence-based guidelines. Multidisciplinary discussion to develop and refine trial design is important for procuring better quality of RCTs of adjuvant chemotherapy of gastric cancer.