Data formats and standards for opportunistic rainfall sensors

Open Res Eur. 2024 Feb 13:3:169. doi: 10.12688/openreseurope.16068.2. eCollection 2023.

Abstract

Opportunistic sensors are increasingly used for rainfall measurement. However, their raw data are collected by a variety of systems that are often not primarily intended for rainfall monitoring, resulting in a plethora of different data formats and a lack of common standards. This hinders the sharing of opportunistic sensing (OS) data, their automated processing, and, at the end, their practical usage and integration into standard observation systems. This paper summarises the experiences of the more than 100 members of the OpenSense Cost Action involved in the OS of rainfall. We review the current practice of collecting and storing precipitation OS data and corresponding metadata, and propose new common guidelines describing the requirements on data and metadata collection, harmonising naming conventions, and defining human-readable and machine readable file formats for data and metadata storage. We focus on three sensors identified by the OpenSense community as prominent representatives of the OS of precipitation: Commercial microwave links (CML): fixed point-to-point radio links mainly used as backhauling connections in telecommunication networks Satellite microwave links (SML): radio links between geostationary Earth orbit (GEO) satellites and ground user terminals. Personal weather stations (PWS): non-professional meteorological sensors owned by citizens. The conventions presented in this paper are primarily designed for storing, handling, and sharing historical time series and do not consider specific requirements for using OS data in real time for operational purposes. The conventions are already now accepted by the ever growing OpenSense community and represent an important step towards automated processing of OS raw data and community development of joint OS software packages.

Keywords: commercial microwave links; data format; data standards; naming conventions; opportunistic rainfall sensing; personal weather stations; satelllite microwave links.

Plain language summary

Opportunistic sensors, devices primarily intended not intended for sensing, are increasingly used for rainfall measurement. The lack of conventions defining which data should be stored and how, makes it difficult to automatically process the data and integrate these observations into standard monitoring networks. This paper reviews current practice of collecting and storing precipitation opportunistic sensing (OS) data based on the experience of more than 100 members of the OpenSense Cost Action and suggest common data format standards. We focus on three sensors identified by the OpenSense community as prominent representatives of the OS of precipitation: Commercial microwave links (CML), Satellite Microwave Links (SML), and Personal Weather Stations (PWS). The conventions are already now accepted by the ever growing OpenSense community and represent an important step towards automated processing of OS raw data and community development of joint OS software packages.

Grants and funding

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No CA20136 (Opportunistic precipitation sensing network [OpenSense]).