Introduction: For accurate diagnosis, inter-observer agreement of criteria is important.
Methods: Using case records, the reproducibility of the original and revised El Escorial diagnostic criteria for amyotrophic lateral sclerosis were tested in a consecutive series of people referred to the Scottish Motor Neuron Disease Register.
Results: Agreement between independent observers was similar (weighted kappa: 0.783, 95% CI 0.656 to 0.911 (original criteria), 0.681, 95% CI 0.485 to 0.878 (revised)).
Conclusions: Serious errors are unlikely, but the revised criteria may be less reproducible as they include more diagnostic categories. Revisions of diagnostic criteria should be tested for reproducibility and validity prior to widespread adoption.