Computer vision for assessing species color pattern variation from web-based community science images

Maggie M Hantak; Robert P Guralnick; Alina Zare; Brian J Stucky

doi:10.1016/j.isci.2022.104784

Computer vision for assessing species color pattern variation from web-based community science images

iScience. 2022 Jul 19;25(8):104784. doi: 10.1016/j.isci.2022.104784. eCollection 2022 Aug 19.

Authors

Maggie M Hantak¹, Robert P Guralnick¹, Alina Zare², Brian J Stucky^{1

3}

Affiliations

¹ Florida Museum of Natural History, University of Florida, Gainesville, FL 32611, USA.
² Department of Electrical, and Computer Engineering, University of Florida, Gainesville, FL 32611, USA.
³ Agricultural Research Service, U.S. Department of Agriculture, Beltsville, MD 20705, USA.

Abstract

Openly available community science digital vouchers provide a wealth of data to study phenotypic change across space and time. However, extracting phenotypic data from these resources requires significant human effort. Here, we demonstrate a workflow and computer vision model for automatically categorizing species color pattern from community science images. Our work is focused on documenting the striped/unstriped color polymorphism in the Eastern Red-backed Salamander (Plethodon cinereus). We used an ensemble convolutional neural network model to analyze this polymorphism in 20,318 iNaturalist images. Our model was highly accurate (∼98%) despite image heterogeneity. We used the resulting annotations to document extensive niche overlap between morphs, but wider niche breadth for striped morphs at the range-wide scale. Our work showcases key design principles for using machine learning with heterogeneous community science image data to address questions at an unprecedented scale.

Keywords: Computer science; Ecology; Evolutionary biology.