Revisiting crowd behaviour analysis through deep learning: Taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects

Francisco Luque Sánchez; Isabelle Hupont; Siham Tabik; Francisco Herrera

doi:10.1016/j.inffus.2020.07.008

Revisiting crowd behaviour analysis through deep learning: Taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects

Inf Fusion. 2020 Dec:64:318-335. doi: 10.1016/j.inffus.2020.07.008. Epub 2020 Jul 29.

Authors

Francisco Luque Sánchez¹, Isabelle Hupont², Siham Tabik¹, Francisco Herrera¹

Affiliations

¹ Andalusian Research Institute in Data Science and Computational Intelligence, University of Granada, Granada, Spain.
² Herta Security, Barcelona, Spain.

Abstract

Crowd behaviour analysis is an emerging research area. Due to its novelty, a proper taxonomy to organise its different sub-tasks is still missing. This paper proposes a taxonomic organisation of existing works following a pipeline, where sub-problems in last stages benefit from the results in previous ones. Models that employ Deep Learning to solve crowd anomaly detection, one of the proposed stages, are reviewed in depth, and the few works that address emotional aspects of crowds are outlined. The importance of bringing emotional aspects into the study of crowd behaviour is remarked, together with the necessity of producing real-world, challenging datasets in order to improve the current solutions. Opportunities for fusing these models into already functioning video analytics systems are proposed.

Keywords: Crowd anomaly detection; Crowd behaviour analysis; Crowd emotions; Deep learning; Models fusion; Review.