One hundred and nine adults were screened in the community using the abridged version of the CIDI (CIDIS). The subjects comprised DSM-III-R current cases (N = 48), lifetime cases (N = 31) and non-cases (N = 30). The interviews with the 109 subjects were conducted by one of two pairs of clinicians and videotaped. Each interviewer-pair included a psychiatrist and a clinical psychologist. They rated the community version of the Needs for Care (NFCAS-C) by consensus. The other pair of judges then viewed the video and rated the NFCAS-C independently. The agreement on overall needs was excellent (kappa = 0.75), and very good for four of the seven specific sections (from kappa = 0.61 to 0.81). One section could not be rated because of low prevalence, and agreement was less good for the remaining two sections. Agreement was good on specific interventions (medication, kappa = 0.60; specific psychotherapy, kappa = 0.55), but poor on non-specific interventions. The majority of disagreements were due to differences in clinical judgement rather than to technical errors. A new instruction manual has been produced and should help training as well as stabilizing reliability. In devising reliable and valid instruments based on clinical judgement, a balance must be achieved between enhancing reliability with more precise rules and constraining clinical judgement so tightly that validity is lost.