The literature that is relevant to evaluation of treatment effectiveness is large, scattered and difficult to assemble for appraisal. This scoping review first develops a conceptual framework to help organize the field, and second, uses the framework to appraise early psychosis intervention (EPI) studies. Literature searches were used to identify representative study designs, which were then sorted according to evaluation approach. The groupings provided a conceptual framework upon which a map of the field could be drawn. Key words were cross-checked against definitions in dictionaries of scientific terms and the National Library of Medicine Medical Subject Headings (MeSH) browser. Using the final list of key words as search terms, the EPI evaluation literature was appraised. Experimental studies could be grouped into two classes: efficacy and effectiveness randomized controlled trials. Non-experimental studies could be subgrouped into at least four overlapping categories: clinical epidemiological; health service evaluations; quality assurance studies; and, quasi-experimental assessments of treatment effects. Applying this framework to appraise EPI studies indicated promising evidence for the effectiveness of EPI irrespective of study design type, and a clearer picture of where future evaluation efforts should be focused. Reliance on clinical trials alone will restrict the type of information that can inform clinical practice. There is convergent evidence for the benefits of specialized EPI service functions across a range of study designs. Greater investment in health services research and quality assurance approaches in evaluating EPI effectiveness should be made, which will involve scaling up of study sizes and development of an EPI programme fidelity rating template. The degree of complexity of the evaluation field suggests that greater focus on research methodology in the training of Australasian psychiatrists is urgently needed.