EXSCLAIM!: Harnessing materials science literature for self-labeled microscopy datasets

Eric Schwenker; Weixin Jiang; Trevor Spreadbury; Nicola Ferrier; Oliver Cossairt; Maria K Y Chan

doi:10.1016/j.patter.2023.100843

EXSCLAIM!: Harnessing materials science literature for self-labeled microscopy datasets

Patterns (N Y). 2023 Sep 30;4(11):100843. doi: 10.1016/j.patter.2023.100843. eCollection 2023 Nov 10.

Authors

Eric Schwenker^{1

2}, Weixin Jiang^{1

3}, Trevor Spreadbury^{1

3}, Nicola Ferrier⁴, Oliver Cossairt³, Maria K Y Chan¹

Affiliations

¹ Center for Nanoscale Materials, Argonne National Laboratory, Argonne, IL 60439, USA.
² Department of Materials Science and Engineering, Northwestern University, Evanston, IL 60208, USA.
³ Department of Computer Science, Northwestern University, Evanston, IL 60208, USA.
⁴ Mathematics and Computer Science, Argonne National Laboratory, Argonne, IL 60439, USA.

Abstract

This work introduces the EXSCLAIM! toolkit for the automatic extraction, separation, and caption-based natural language annotation of images from scientific literature. EXSCLAIM! is used to show how rule-based natural language processing and image recognition can be leveraged to construct an electron microscopy dataset containing thousands of keyword-annotated nanostructure images. Moreover, it is demonstrated how a combination of statistical topic modeling and semantic word similarity comparisons can be used to increase the number and variety of keyword annotations on top of the standard annotations from EXSCLAIM! With large-scale imaging datasets constructed from scientific literature, users are well positioned to train neural networks for classification and recognition tasks specific to microscopy-tasks often otherwise inhibited by a lack of sufficient annotated training data.