Validation of a form for assessing the professional performance of residents in cardiology by nurses

Abstract

Background: There is a strong need for transformation in our assessment systems from one that evaluates performance based on levels of training to another that focuses on professional competence to meet the expected requirements for the practice of the profession. The aim of this study is to validate for the first time a Spanish version of a new tool for assessing the professional performance of residents by nurses newly developed in the Ottawa Hospital (O-RON).

Materials and methods: After the author's written authorization, the original O-RON form was translated and cross-culturally adapted. Then we conducted a prospective observational study in two cardiology centers in the city of Buenos Aires. The validity of the tools was evaluated by the ability of the instrument to discriminate the level of experience of the residents according to their post-graduate year level. Data is expressed as percentages and frequencies of the qualifications obtained in the different questions. The chi-square test was used to assess the significance of the differences obtained. A generalizability test was used to evaluate reliability. Feasibility was defined as a minimum of 4 assessments per resident per evaluation round. Satisfaction of evaluators was assessed using a survey with a 10-point scale designed by the authors.

Results: A total of 838 evaluations were performed. Regarding validity, the 15-item form could significantly discriminate the experience of the residents according to their postgraduate year level (P < 0,005). Thirty evaluations per resident are required to obtain reliable results. The tool is feasible to implement and an average of 4.55 assessment per resident per evaluation round were achieved throughout the entire experience. This value remained stable during the 8 rounds (1st: 4.65; 2nd: 4.34; 3^rd: 4.47; 4^th: 6.17; 5^th: 4.56; 6^th: 4.08; 7^th: 4.36; 8^th: 3.91). The levels of satisfaction among the evaluators were acceptable.

Conclusion: The Spanish version of the O-RON form can provide residents with a valuable source of feedback from the eyes of nurses on important aspects of their professional training. This tool, positively assessed by the raters, significantly discriminates residents' experience. Its implementation is feasible in our environment, and it is user-friendly, though it requires a considerable number of assessments to achieve high reliability.

Keywords: Cardiology; feedback; nurses; observation.