Study design: An end-user response survey and assessments of inter-rater reliability before and after training.
Objectives: Evaluate the spinal cord injury (SCI) application of the international classification of external cause of injury (ICECI) in a mixed group of untrained and trained coders to assess agreement, refine coding and training methodology.
Setting: An interactive coding workshop for an international group of coders with varying previous training.
Methods: Evaluate content validity (qualitative survey) and inter-rater reliability (kappa estimate of agreement) of the ICECI in a variety of injury scenarios presented within a computerized data-entry and training module. The results of this evaluation are compared with an earlier published gold standard.
Results: The ICECI is a flexible data coding system that appears to work with reasonable content validity in the regions assessed with English-language coders. Training appeared to narrow the difference between the inexperienced and trained coders. This is reflected in a borderline tendency for lower kappa scores pre-training compared with an earlier examined group of expert coders (P=0.073) but no difference in kappa scores after training (P=0.67). Computer-based training on a face-to-face level with computerized data entry appears an effective tool for training coders to use the ICECI.
Conclusions: This report shows that using electronic data-entry and training assistance, inexperienced coders using the SCI-ICECI computerized system quickly approach the levels of agreement of trained coders in related data systems. The content validity of the training data set is adequate but needs to include more cases representative for use in SCI.