Assessing the Performance of a Classification-Based Vulnerability Analysis Model

Tai-ran Wang; Vincent Mousseau; Nicola Pedroni; Enrico Zio

doi:10.1111/risa.12305

Assessing the Performance of a Classification-Based Vulnerability Analysis Model

Risk Anal. 2015 Sep;35(9):1674-89. doi: 10.1111/risa.12305. Epub 2014 Dec 8.

Authors

Tai-ran Wang¹, Vincent Mousseau², Nicola Pedroni¹, Enrico Zio^{1

3}

Affiliations

¹ Chair on Systems Science and the Energy Challenge, European Foundation for New Energy-Electricité de France, Ecole Centrale Paris and Supélec, Chatenay Malabry Cedex, France.
² Laboratory of Industrial Engineering, Ecole Centrale Paris, Grande Voie des Vignes, F92-295, Chatenay Malabry Cedex, France.
³ Politecnico di Milano, Energy Department, Nuclear Section, c/o Cesnef, via Ponzio 33/A, 20133, Milan, Italy.

PMID: 25487957
DOI: 10.1111/risa.12305

Abstract

In this article, a classification model based on the majority rule sorting (MR-Sort) method is employed to evaluate the vulnerability of safety-critical systems with respect to malevolent intentional acts. The model is built on the basis of a (limited-size) set of data representing (a priori known) vulnerability classification examples. The empirical construction of the classification model introduces a source of uncertainty into the vulnerability analysis process: a quantitative assessment of the performance of the classification model (in terms of accuracy and confidence in the assignments) is thus in order. Three different app oaches are here considered to this aim: (i) a model-retrieval-based approach, (ii) the bootstrap method, and (iii) the leave-one-out cross-validation technique. The analyses are presented with reference to an exemplificative case study involving the vulnerability assessment of nuclear power plants.

Keywords: Classification model; MR-Sort; confidence estimation; nuclear power plants; vulnerability analysis.