Unifying Multi-site MRI Harmonization Via Content-Style Representation Disentanglement

Published in Computational Sciences
Unifying Multi-site MRI Harmonization Via Content-Style Representation Disentanglement

Share this post

Choose a social network to share with, or copy the shortened URL to share elsewhere

This is a representation of how your post may appear on social media. The actual post will vary between social networks

The need for multi-site MRI harmonization

Structural MRI provides detailed high-resolution images of the brain's anatomy for visualizing different types of brain tissue, including gray matter, white matter, and cerebrospinal fluid. This in turn is crucial for identifying the size, shape, and integrity of various brain structures, especially for various brain abnormalities and diseases. Large-scale multi-site MRI studies are increasingly common, allowing much greater sample sizes not possible in the past decades (e.g., n > 10,000) for greater power in detecting subtle changes in association with brain development, aging, and diseases. However, harmonization of imaging data across sites remains a largely unsolved challenge, resulting in non-biological variation across scanners that negatively affects study outcomes.

Retrospective harmonization refers to the process of methodologically adjusting and standardizing MRI data collected from different sites or scanners to ensure consistency and comparability across all sites, aiming to minimize non-biological variability introduced by factors such as different scanner models, manufacturers, software versions, imaging protocols, and environmental conditions across study sites. Traditionally, statistics-driven methods are used to align intensity distributions of multi-site imaging data, but these methods are typically applied on the image globally without catering to variation in fine local details.

Advances in machine learning (ML) are suggesting the possibility of new paradigms in multi-site harmonization using image synthesis. One of the key advantages of ML-based methods, particularly those based on deep learning, is the automatization and standardization of tedious and time-consuming tasks.

Learning harmonization without paired data

Deep learning harmonization methods have been shown to significantly outperform classical methods in dealing with the complexity of anatomical details due to its prominent feature extraction capability. However, a crippling limitation of most deep learning methods to date is that they rely on supervised training and hence require scanning the same subjects (a.k.a. traveling phantoms) multiple times at the different sites. This is apparently very challenging, especially when images in sufficient quantity are needed in practice for reliable model training.

By contrast, unsupervised image-to-image translation techniques, such as CycleGAN, utilize unpaired images, which eliminates the necessity for paired data. However, these methods encounter scalability challenges in extensive multi-site scenarios, as they are designed to learn mappings between pairs of sites. This pair-wise approach is less effective because it fails to leverage the comprehensive information accessible across all sites. Additionally, these unsupervised strategies risk altering anatomical structures in the process of adjusting image contrast.

Our solution: Content-style disentanglement

Our recent Communications Engineering paper "Learning Multi-site Harmonization of Magnetic Resonance Images Without Traveling Human Phantoms" presents a unified deep learning framework for simultaneous multi-site harmonization. Our method, multi-site unsupervised representation disentangler (MURD) treats simultaneous multi-site harmonization as a unified unsupervised image-to-image translation across multiple domains, decomposing an image into anatomical content that is invariant across sites and appearance style (e.g., intensity and contrast) that is dependent on each site. Harmonized images are generated by combining the content of an image with styles specific to the different sites, as illustrated in Figure 1. Our method requires only a small amount of unpaired data for highly efficient training and is therefore a game changer and allows data in a wide range of existing studies, conducted via different imaging protocols, to be harmonized without additional data collection.

Figure 1: (Left) MURD’s content-style disentangled cycle translation between any two sites. (Right) MURD’s capabilities in any-site harmonization, improving volumetric measurements, cross-resolution harmonization, and continuous harmonization.

Potential impact

MURD's ability to harmonize across sites without altering anatomical details could significantly improve the quality and consistency of MRI scans obtained from different locations. Such harmonization is crucial for accurate diagnosis and treatment planning, especially in cases where patients' scans are taken at different facilities. In large-scale multi-site clinical studies, such as those related to brain development or neurodegenerative diseases, ensuring consistent imaging data is essential. MURD allows for the retrospective harmonization of existing study data, enhancing the reliability and accuracy of research outcomes. MURD can also be instrumental in training AI models for medical image analysis as consistent and harmonized data are vital for developing algorithms that are accurate and generalizable across various imaging settings. Additionally, MURD enables the sharing of imaging data for second opinions or specialist consultations without the concern of variability due to differences in imaging equipment, promoting greater collaboration among healthcare providers. Besides, eliminating the need for human phantoms to travel between sites for the purpose of harmonizing MRI data reduces costs and logistical complexities. This efficiency can lead to faster diagnosis and treatment, benefiting both healthcare providers and patients.

Please sign in or register for FREE

If you are a registered user on Research Communities by Springer Nature, please sign in

Follow the Topic

Magnetic Resonance Imaging
Mathematics and Computing > Computer Science > Computer Imaging, Vision, Pattern Recognition and Graphics > Computer Vision > Medical Imaging > Magnetic Resonance Imaging
Machine Learning
Mathematics and Computing > Computer Science > Artificial Intelligence > Machine Learning

Related Collections

With collections, you can get published faster and increase your visibility.

Consumer waste valorization

This collection will focus on chemical, materials, mechanical and other engineering advances tackling post-consumer solid waste streams such as textiles and paper, food, electronics and plastics.

Publishing Model: Open Access

Deadline: May 31, 2024

Cleaner Flight

This joint collection from Communications Engineering, Communications Earth and Environment and Scientific Reports aims to offer insights into the environmental impacts of aviation, along with technological design, operational and infrastructure solutions to mitigate aviation’s future environmental footprint.

Publishing Model: Open Access

Deadline: Apr 16, 2024