To evaluate the quality of the supervision of residents, questionnaires are frequently used.<br/> AIM: Changes in resident ratings were assessed using the validated Evaluation and Feedback for Effective Clinical Teaching (effect) questionnaire.<br/> METHOD: Supervisors (n=89) of nine medical specialities were evaluated using a validated instrument (effect). Mean overall scores (mos) and mean scale scores were compared using paired t-tests. Ten supervisors were interviewed to determine factors that stimulate or hinder improvement after an evaluation.<br/> RESULTS: The strongest increase was seen in supervisors with an mos <4.0. Motivation to improve and dialogue about the feedback stimulated improvement. Lack of time and support hindered improvement.<br/> CONCLUSION: Evaluating supervisors by providing feedback and a dialogue is suitable to improve their supervision of residents.