Seeing the Unseen: HistoTME's AI Vision Reveals Secrets of the Lung Cancer Tumor Microenvironment

Seeing the Unseen: HistoTME's AI Vision Reveals Secrets of the Lung Cancer Tumor Microenvironment
Like

Share this post

Choose a social network to share with, or copy the URL to share elsewhere

This is a representation of how your post may appear on social media. The actual post will vary between social networks

The discovery of immune checkpoint inhibitors (ICI) has revolutionized lung cancer treatment. These powerful drugs unleash our immune system to fight cancer, leading to long-lasting remissions and offering hope where it was once scarce. However, a significant challenge remains: many patients don't experience long-lasting benefits. To truly unlock the potential of ICIs, we need to understand why some patients respond while others don't.  This is where biomarkers come in – they are key to identifying the right treatment for the right patient, paving the way for personalized immunotherapy and better outcomes.

The Challenge: Understanding the Tumor Microenvironment

Currently, doctors often use the presence of a protein called PD-L1 on cancer cells to predict who might benefit from ICIs. But it's not a perfect predictor. Some patients with low PD-L1 levels respond well, while others with high levels don't. Why?  A crucial piece of the puzzle lies within the tumor microenvironment (TME) – the complex ecosystem surrounding the tumor. The TME composition plays a critical role in how a patient responds to treatment, making its comprehensive understanding essential to optimizing patient care and improving clinical outcomes. One can think of the TME like a bustling city. It's filled with various "residents" – not just cancer cells, but also immune cells like T cells and B cells, along with other players like cancer associated fibroblasts and macrophages. These cells constantly interact with each other, influencing the overall immune response against the tumor. Recent advancements in spatial omics and multiplexed imaging allow us to map this "city" in incredible detail, down to the single-cell level. But these methods are expensive and complex, limiting their widespread use in clinics.

Decoding the tumor microenvironment using digital pathology and AI:

Fortunately, there's another readily available resource: Hematoxylin and Eosin (H&E) stained pathology slides. These are routinely used by pathologists to diagnose cancer. While seemingly simple, these slides contain a wealth of information about the TME that often goes unnoticed.  Here's where Artificial Intelligence (AI) comes in. AI-powered algorithms can analyze these everyday images, extracting hidden insights about the TME and potentially uncover new, cost-effective biomarkers for predicting ICI response.

 Introducing HistoTME: a new digital pathology AI tool for profiling of the TME

Our team set out to develop an AI tool that could analyze H&E-stained whole slide images (WSIs) from patients with non-small cell lung cancer (NSCLC) to identify TME features that predict response to ICIs. WSIs are massive (~40,000x40,000 pixels) and contain diverse tissue features. We had limited data to train a traditional, end-to-end AI model, and interpreting its findings would be difficult. This motivated us to find a new, more interpretable way to represent these images, one that specifically captures TME features. This challenge led to the creation of HistoTME, a novel deep learning tool that can predict the cellular and molecular composition of the TME directly from a standard H&E-stained WSI. What sets HistoTME apart? Unlike conventional computational pathology methods, our approach centers on the use of self-supervised foundation models, a cutting-edge technique in digital pathology. These models are trained on massive datasets of unlabeled WSIs, learning to extract meaningful representations of histological features. HistoTME cleverly repurposes these pre-trained models to analyze new images, employing an attention-based learning algorithm to pinpoint areas critical for predicting expression of specific gene signatures.

This figure depicts a schematic diagram of how HistoTME's AI algorithm works. A whole slide H&E image is provided as input and broken down to 256x256 µm tiles. Each image tile captures a unique region of the WSI. The UNI foundation model is applied to each tile to extract rich histopathological feature embeddings. Finally a multitask attention based multiple instance learning module learns to summarize relevant features from the WSI that correlate with the expression of specific molecular signatures 

This novel combination of self-supervised pre-training and attention-based learning allows us to accurately predict the abundance of various immune cell types within the tumor microenvironment, such as T cells, B cells, NK cells, and macrophages, without relying on labor-intensive manual annotations! It also provides valuable insights into the activity of crucial microenvironment-related pathways, such as tumor antigen presentation, proliferation, epithelial-to-mesenchymal transition, and interferon gamma signaling, which are difficult to assess using conventional methods. Importantly, HistoTME's high throughput TME analysis improves the identification of NSCLC patients who will benefit from ICI treatment, independent of traditional biomarkers like PD-L1 and TILs. This is clinically significant because our tool can help doctors identify patients with cancers that are completely resistant to ICIs and thus should be spared from the associated adverse effects and costs.

 Conclusion:

In summary, HistoTME represents a significant step forward in our quest to personalize cancer immunotherapy treatment. By leveraging readily available histopathology data and advanced AI, we can gain deeper insights into the complexities of the tumor microenvironment, without the need for sophisticated molecular staining or imaging procedures. These crucial insights can be harnessed to design more effective treatment strategies, and ultimately, build a brighter future for those battling lung cancer. This work would not have been possible without the help of my co-author Alex Chen, who helped develop this ground-breaking AI tool and the guidance and feedback of our collaborators, especially Dr. Tamara Jamaspishvili!

Please sign in or register for FREE

If you are a registered user on Research Communities by Springer Nature, please sign in

Follow the Topic

Artificial Intelligence
Mathematics and Computing > Computer Science > Artificial Intelligence
Cancer Immunotherapy
Life Sciences > Biological Sciences > Cancer Biology > Cancer Therapy > Cancer Immunotherapy
Computational Biology
Mathematics and Computing > Mathematics > Applications of Mathematics > Computational Biology
Pathology
Life Sciences > Health Sciences > Clinical Medicine > Pathology

Related Collections

With collections, you can get published faster and increase your visibility.

Noninvasive biomarkers: bridging real-world evidence and clinical trial data

This collection seeks to advance the discussion on noninvasive biomarkers by exploring their applications and implications across different data landscapes, namely real-world data (RWD) and clinical trial data.

Publishing Model: Open Access

Deadline: Sep 13, 2025

Antibody-drug conjugates for solid tumors

Antibody drug conjugates (ADCs) have revolutionized the field of oncology with more than 100 ADCs in clinical development for different solid tumors. In this series, we will comprehensively review the clinical development of ADCs in different solid tumors, and how ADCs are replacing traditional chemotherapy in oncology.

Publishing Model: Open Access

Deadline: Jul 15, 2025