This study evaluates different methods of assessing psychotherapy skills. Nine therapists were evaluated in the following ways: (1) didactic examination, (2) global ratings by trainers, (3) supervisor's ratings based on therapist's retrospective report of therapy sessions in supervision, (4) therapists' self-ratings, and (5) independent evaluators' ratings of videotaped psychotherapy sessions. Results show poor agreement among assessments of therapists' skills based on different data sources. Most important, ratings based on review of videotaped sessions were uncorrelated with those based on supervisor's discussion of process material with the therapist. Of the five types of ratings of psychotherapist skill, only the supervisor's ratings were correlated with patient outcome. The implications of these findings are discussed in terms of therapist assessment procedures used in clinical training programs.