Reliability and validity of an employer-completed safety hazard and management assessment questionnaire

Abstract

Introduction: Managing and improving occupational safety and health requires evaluating performance. Organizations are encouraged to use both lagging indicators (such as injury rates and costs) and leading indicators (such as questionnaire-assessed safety hazards and management practices) for this purpose, but the association between types of indicators over time can be complex. Longitudinal data can assist in clarifying these associations and increasing indicator utility.

Method: Employer data were used to evaluate the reliability and predictive validity of a safety management questionnaire. Employers' longitudinal questionnaire responses and workers' compensation (WC) claims data were analyzed using a marginal model with time-dependent covariates. Multivariable Poisson and linear regression analyses with claim rate and logarithmic cost, respectively, as dependent variables were carried out after adjusting for industry sector and size. Questionnaire data were used to evaluate questionnaire scaling properties and to assess generalizability of results.

Results: One safety management scale was associated with a better WC outcome as predicted and two scales were unexpectedly associated with poorer WC claim outcomes. Analyses assisted in interpreting the latter results, suggesting that WC outcomes were a stimulus for change in some cases. Twelve hazards assessed on the questionnaire were associated with poorer WC claim outcomes as predicted.

Conclusions: This study extends leading indicator research using longitudinal questionnaire and WC claims data from employers. Analyses provided insight into associations between leading and lagging indicators, emphasizing the importance of both for safety improvement. Safety management questionnaire scales were predictive of WC claim outcomes, although support for hazard assessments as leading indicators was stronger.

Practical applications: This study supports the use of employer-completed hazard assessment questionnaires for targeting and prioritizing improvement efforts. Employer-completed safety management scales may be useful for directing improvement efforts, although the conditions under which they are completed, including submission to insurers, require additional consideration.

Keywords: Lagging metrics; Leading; Predictor; Workers’ compensation.