Quantifying noise limitations of neural network segmentations in high-resolution transmission electron microscopy

Matthew Helmi Leth Larsen; William Bang Lomholdt; Cuauhtemoc Nuñez Valencia; Thomas W Hansen; Jakob Schiøtz

doi:10.1016/j.ultramic.2023.113803

Quantifying noise limitations of neural network segmentations in high-resolution transmission electron microscopy

Ultramicroscopy. 2023 Nov:253:113803. doi: 10.1016/j.ultramic.2023.113803. Epub 2023 Jul 13.

Authors

Matthew Helmi Leth Larsen¹, William Bang Lomholdt², Cuauhtemoc Nuñez Valencia¹, Thomas W Hansen², Jakob Schiøtz³

Affiliations

¹ Computational Atomic-scale Materials Design (CAMD), Department of Physics, Technical University of Denmark, DK-2800 Kgs. Lyngby, Denmark.
² National Center for Nano Fabrication and Characterization, Technical University of Denmark, DK-2800 Kgs. Lyngby, Denmark.
³ Computational Atomic-scale Materials Design (CAMD), Department of Physics, Technical University of Denmark, DK-2800 Kgs. Lyngby, Denmark. Electronic address: schiotz@fysik.dtu.dk.

PMID: 37499574
DOI: 10.1016/j.ultramic.2023.113803

Abstract

Motivated by the need for low electron dose transmission electron microscopy imaging, we report the optimal frame dose (i.e.e^-/Å²) range for object detection and segmentation tasks with neural networks. The MSD-net architecture shows promising abilities over the industry standard U-net architecture in generalising to frame doses below the range included in the training set, for both simulated and experimental images. It also presents a heightened ability to learn from lower dose images. The MSD-net displays mild visibility of a Au nanoparticle at 20-30 e^-/Å², and converges at 200 e^-/Å² where a full segmentation of the nanoparticle is achieved. Between 30 and 200 e^-/Å² object detection applications are still possible. This work also highlights the importance of modelling the modulation transfer function when training with simulated images for applications on images acquired with scintillator based detectors such as the Gatan Oneview camera. A parametric form of the modulation transfer function is applied with varying ranges of parameters, and the effects on low electron dose segmentation is presented.

Keywords: Beam damage; HR-TEM; Machine learning; Modulation transfer function; Signal-to-noise.