Deep learning with self-supervision and uncertainty regularization to count fish in underwater images

PLoS One. 2022 May 4;17(5):e0267759. doi: 10.1371/journal.pone.0267759. eCollection 2022.

Abstract

Effective conservation actions require effective population monitoring. However, accurately counting animals in the wild to inform conservation decision-making is difficult. Monitoring populations through image sampling has made data collection cheaper, wide-reaching and less intrusive but created a need to process and analyse this data efficiently. Counting animals from such data is challenging, particularly when densely packed in noisy images. Attempting this manually is slow and expensive, while traditional computer vision methods are limited in their generalisability. Deep learning is the state-of-the-art method for many computer vision tasks, but it has yet to be properly explored to count animals. To this end, we employ deep learning, with a density-based regression approach, to count fish in low-resolution sonar images. We introduce a large dataset of sonar videos, deployed to record wild Lebranche mullet schools (Mugil liza), with a subset of 500 labelled images. We utilise abundant unlabelled data in a self-supervised task to improve the supervised counting task. For the first time in this context, by introducing uncertainty quantification, we improve model training and provide an accompanying measure of prediction uncertainty for more informed biological decision-making. Finally, we demonstrate the generalisability of our proposed counting framework through testing it on a recent benchmark dataset of high-resolution annotated underwater images from varying habitats (DeepFish). From experiments on both contrasting datasets, we demonstrate our network outperforms the few other deep learning models implemented for solving this task. By providing an open-source framework along with training data, our study puts forth an efficient deep learning template for crowd counting aquatic animals thereby contributing effective methods to assess natural populations from the ever-increasing visual data.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Animals
  • Benchmarking
  • Deep Learning*
  • Ecosystem
  • Fishes
  • Uncertainty

Grants and funding

This work has been partially supported by the Spanish project PID2019-105093GB-I00 (MINECO/FEDER, UE) and CERCA Programme/Generalitat de Catalunya (https://portal.mineco.gob.es), and by ICREA under the ICREA Academia programme (https://www.icrea.cat/) awarded to S.G. The data sampling was supported by research grants from the National Geographic Society (Discovery Grant WW210R-17) and the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES Brazil; https://www.gov.br/capes/pt-br; #88881.170254/2018-01) and Conselho Nacional de Pesquisa e Desenvolvimento Tecnológico (CNPq Brazil; https://www.gov.br/cnpq/pt-br; #153797/2016-9) granted to M.C. M.C. is supported by The Max Planck Society via the Department for the Ecology of Animal Societies at the Max Planck Institute of Animal Behaviour (https://www.ab.mpg.de/crofoot), and grants from the CAPES-DAAD PROBRAL Research Programme (#23038.002643/2018-01; (https://www.daad.de/en/)) and the SELA CNPq-PELD Research Program (SELA 445301/2020-1). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.