DARLENE - Improving situational awareness of European law enforcement agents through a combination of augmented reality and artificial intelligence solutions

Konstantinos C Apostolakis; Nikolaos Dimitriou; George Margetis; Stavroula Ntoa; Dimitrios Tzovaras; Constantine Stephanidis

doi:10.12688/openreseurope.13715.2

DARLENE - Improving situational awareness of European law enforcement agents through a combination of augmented reality and artificial intelligence solutions

Open Res Eur. 2022 Jan 21:1:87. doi: 10.12688/openreseurope.13715.2. eCollection 2021.

Authors

Konstantinos C Apostolakis¹, Nikolaos Dimitriou², George Margetis¹, Stavroula Ntoa¹, Dimitrios Tzovaras², Constantine Stephanidis^{1

3}

Affiliations

¹ Institute of Computer Science, Foundation for Research and Technology Hellas, Heraklion, Crete, GR-70013, Greece.
² Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, GR-57001, Greece.
³ Computer Science Department, University of Crete, Heraklion, GR-70013, Greece.

Abstract

Background: Augmented reality (AR) and artificial intelligence (AI) are highly disruptive technologies that have revolutionised practices in a wide range of domains, including the security sector. Several law enforcement agencies (LEAs) employ AI in their daily operations for forensics and surveillance. AR is also gaining traction in security, particularly with the advent of affordable wearable devices. Equipping police officers with the tools to facilitate an elevated situational awareness (SA) in patrolling and tactical scenarios is expected to improve LEAs' safety and capacity to deliver crucial blows against terrorist and/or criminal threats. Methods: In this paper we present DARLENE, an ecosystem incorporating novel AI techniques for activity recognition and pose estimation tasks, combined with a wearable AR framework for visualization of the inferenced results via dynamic content adaptation according to the wearer's stress level and operational context. The concept has been validated with end-users through co-creation workshops, while the decision-making mechanism for enhancing LEAs' SA has been assessed with experts. Regarding computer vision components, preliminary tests of the instance segmentation method for humans' and objects' detection have been conducted on a subset of videos from the RWF-2000 dataset for violence detection, which have also been used to test a human pose estimation method that has so far exhibited impressive results, constituting the basis of further developments in DARLENE. Results: Evaluation results highlight that target users are positive towards the adoption of the proposed solution in field operations, and that the SA decision-making mechanism produces highly acceptable outcomes. Evaluation of the computer vision components yielded promising results and identified opportunities for improvement. Conclusions: This work provides the context of the DARLENE ecosystem and presents the DARLENE architecture, analyses its individual technologies, and demonstrates preliminary results, which are positive both in terms of technological achievements and user acceptance of the proposed solution.

Keywords: 5G network; Augmented reality glasses; Computer vision; Control and surveillance of areas; Deep learning; Fight against criminality; Situational awareness.

Grants and funding

This research was financially supported by the European Union’s Horizon 2020 research and innovation programme under the grant agreement No 883297 (project DARLENE). This paper reflects only the authors’ view and the Commission is not responsible for any use that may be made of the information it contains.