Evidence-based medicine is an approach to improved patient care that integrates clinical experience with basic science and clinical research in diagnostic accuracy, prognostic indicators, efficacy, and safety of treatments. Both clinical experience and methodical research assessments are essential components in this type of medical practice and underscore the importance of providing residents the opportunity to gain clinical experience as well as training them in how to perform, apply, and interpret clinical research in diagnostic imaging. The challenge for researchers is to design a study so that the data are valid and may be generalized to clinical situations where the test will be used. When assisting residents in the design of a research project for accuracy assessment of an imaging test, we consistently have observed three problem areas that if uncorrected would preclude the study results from being generalized to clinical situations where the test will be used: (1) understanding what is being measured, (2) appropriate selection of the sample population, and (3) the impact of the variability of the decision criterion. In this paper, we review these issues and suggest some solutions.