Reacting to an emergency requires quick decisions under stressful and dynamic conditions. To react effectively, responders need to know the right actions to take given the risks posed by the emergency. While existing research on risk scales focuses primarily on decision making in static environments with known risks, these scales may be inappropriate for conditions where the decision maker's time and mental resources are limited and may be infeasible if the actual risk probabilities are unknown. In this article, we propose a method to develop context-specific, scenario-based risk scales designed for emergency response training. Emergency scenarios are used as scale points, reducing our dependence on known probabilities; these are drawn from the targeted emergency context, reducing the mental resources required to interpret the scale. The scale is developed by asking trainers/trainees to rank order a range of risk scenarios and then aggregating these orderings using a Kemeny ranking. We propose measures to assess this aggregated scale's internal consistency, reliability, and validity, and we discuss how to use the scale effectively. We demonstrate our process by developing a risk scale for subsurface coal mine emergencies and test the reliability of the scale by repeating the process, with some methodological variations, several months later.
© 2012 Society for Risk Analysis.