Multi-resolution deep learning characterizes tertiary lymphoid structures and their prognostic relevance in solid tumors

Mart van Rijthoven; Simon Obahor; Fabio Pagliarulo; Maries van den Broek; Peter Schraml; Holger Moch; Jeroen van der Laak; Francesco Ciompi; Karina Silina

doi:10.1038/s43856-023-00421-7

Multi-resolution deep learning characterizes tertiary lymphoid structures and their prognostic relevance in solid tumors

Commun Med (Lond). 2024 Jan 5;4(1):5. doi: 10.1038/s43856-023-00421-7.

Authors

Affiliations

¹ Pathology Department, Radboud University Medical Center, Nijmegen, Netherlands. mart.vanrijthoven@radboudumc.nl.
² Institute of Experimental Immunology, University of Zurich, Zurich, Switzerland.
³ Department of Pathology and Molecular Pathology, University Hospital Zurich, Zurich, Switzerland.
⁴ Faculty of Medicine, University of Zurich, Zurich, Switzerland.
⁵ Pathology Department, Radboud University Medical Center, Nijmegen, Netherlands.
⁶ Institute of Pharmaceutical Sciences, Swiss Federal Institute of Technology, Zurich, Switzerland.

^# Contributed equally.

Abstract

Background: Tertiary lymphoid structures (TLSs) are dense accumulations of lymphocytes in inflamed peripheral tissues, including cancer, and are associated with improved survival and response to immunotherapy in various solid tumors. Histological TLS quantification has been proposed as a novel predictive and prognostic biomarker, but lack of standardized methods of TLS characterization hampers assessment of TLS densities across different patients, diseases, and clinical centers.

Methods: We introduce an approach based on HookNet-TLS, a multi-resolution deep learning model, for automated and unbiased TLS quantification and identification of germinal centers in routine hematoxylin and eosin stained digital pathology slides. We developed HookNet-TLS using n = 1019 manually annotated TCGA slides from clear cell renal cell carcinoma, muscle-invasive bladder cancer, and lung squamous cell carcinoma.

Results: Here we show that HookNet-TLS automates TLS quantification across multiple cancer types achieving human-level performance and demonstrates prognostic associations similar to visual assessment.

Conclusions: HookNet-TLS has the potential to be used as a tool for objective quantification of TLS in routine H&E digital pathology slides. We make HookNet-TLS publicly available to promote its use in research.

Plain language summary

Tertiary lymphoid structures (TLS) are dense accumulations of immune cells within a cancer. They have been associated with patient survival and treatment effectiveness. Quantification of TLS in cancer microscopy images may therefore aid clinical decision-making. However, no consensus for defining TLS in such images exists leading to inconsistent and variable findings across different labs and studies. We developed a computational tool for automated and objective TLS quantification in cancer images. The tool, called HookNet-TLS, integrates information from multiple image resolutions, which resembles the process of how a pathologist would identify these structures using a microscope. HookNet-TLS detected TLS similarly to trained researchers in three different tumor types. We provided access to HookNet-TLS to facilitate its development and use for TLS assessment in clinical decision-making and research into the role of TLS in cancer.