Evidence-based psychiatry seeks the best research evidence for use in patient care. Recent research suggests that problems with accuracy, precision, bias, and other sources of unreliability potentially interfere with the validity of psychiatry's evidence base. Because many negative clinical research studies go unpublished, awareness and fuller understanding of these problems are blocked by lack of access to relevant data. Based on the importance of scientific soundness of neuropsychiatric research and patient care, we argue for increased attentiveness by investigators and practitioners to how clinical trials (CTs) interdependently estimate the efficacy of treatments and the effectiveness of methods as fair tests of efficacy. Deference by CT investigators to real-world practice conditions at research sites because of the unreliability introduced into data by these practices does not ensure unbiased evaluations of treatment efficacy. We argue for more systematic attention to sources of unreliability in CT investigations and increased commitments to assure the validity of the neuropsychiatric evidence base. These recommendations aim to determine neuropsychiatric drug efficacy with greater certainty to better quantify the clinical importance of drug-associated effects and to provide CT-evidenced guidance for practitioners to most effectively use drug efficacy in patient care.