Multiple Sclerosis is an inflammatory and degenerative disease of the central nervous system. Great strides in therapy over the past two decades have dramatically improved the clinical trajectory of the disease, which can result in significant physical and cognitive disability when sub-optimally treated.   There are now >15 available disease modifying therapies (DMT) that ameliorate the immune dysregulation that characterizes MS, thereby reducing the formation of new zones of inflammatory demyelination (lesions) in the brain and spinal cord.

The Role of MRI in Monitoring People with MS

Magnetic Resonance Imaging (MRI) is a crucial tool both for the diagnosis of MS (where it is incorporated into formal diagnostic criteria) and for monitoring disease status and response to therapy.  Critically, MRI is capable of detecting new and enlarging lesions in patients whose disease is clinically quiescent.  Why is this important?  The uncontrolled formation of new lesions, even if asymptomatic in the acute phase, can result in the transection of millions of axons in both white and gray matter,  setting the scene for clinical progression and the accrual of disability in later stages of the disease.  As inflammatory activity is usually most intense in the first several years of the disease, it is especially important to rapidly tailor DMT to ensure optimal suppression of new lesion formation  during this phase.

MRI is usually performed at least annually in people with MS, affording the clinician an opportunity to capture the development of subclinical pathology and escalate or change treatment rapidly.   However, advances in MRI technology, which generates 1000s of images for a single brain scan, have outpaced supply in the radiology workforce, highlighting an unmet need for automation or solutions that support the radiologist to improve reporting accuracy and productivity.  Separately, standard radiology reports are necessarily qualitative (or semi-quantitative at best), hindering precision approaches to management that are based in part on MRI.

Translating Advances in AI to the Clinic

Borrowing from advances in computer vision, machine learning (and in particular deep learning) techniques have shown great promise in both segmentation and classification tasks using medical images, generating a proliferation of algorithms for both diagnosis of disease and quantitation of pathological features, especially in the field of neurology.  However, only a small fraction of these have been translated to clinical use, illustrating the many hurdles that must be overcome to transform a published algorithm into a functioning, approved medical device.  Indeed, even approved or certified AI-based medical devices, including those designed to quantitate abnormalities or brain structures in people with MS, have a paucity of clinical validation data.  We sought to redress this deficit in a cohort of 397  multi-center MRI scan pairs acquired from people with MS in routine practice, using iQ-Solutions^TM (Sydney Neuroimaging Analysis Centre, Sydney), which analyses brain MRI scans in Digital Imaging and Communications in Medicine (DICOM) format using a collection of AI algorithms based on deep neural network technology.  iQ-Solutions^TM produces an MS-specific report that includes cross-sectional and longitudinal whole brain, brain substructure and lesion metrics.  The AI tool returns visualizations of relevant segmentations to the PACS for radiologist review (Figure 1).

For the principal metric of clinical interest, the development of new and enlarging MS lesions, we demonstrated superior case-level sensitivity of the AI-based tool over standard radiology reports (93.3% vs 58.3%), relative to a consensus ground truth, with minimal loss of specificity. The excellent performance of the AI tool was primarily driven by failure of the human reporter to capture enlarging lesions consistently, perhaps not an unexpected finding given their visual subtlety.  Notably, a subset of enlarging lesions, namely slowly enlarging lesions (SELs) are under increasing scrutiny as a primary driver of disability worsening in MS, and may be specifically targeted by BTK-inhibitor therapies, several of which are currently in late stage clinical development for treatment of the condition.

We also demonstrated equivalence of iQ-Solutions^TM with a core clinical trial imaging lab for both lesion activity and quantitative brain volumetric measures, including percentage brain volume loss (PBVC), an accepted biomarker of neurodegeneration in MS (mean PBVC -0.32% vs -0.36% respectively), whereas even severe atrophy (>0.8% loss) was not appreciated in radiology reports.  While there are recognised biological factors that confound interpretation at the individual level, quantitation of accelerated brain tissue loss may critically impact clinical decision-making as neuroprotective therapies for MS (and other neurodegenerative therapies) emerge.  

'Experiental' referencing may provide more meaningful interpretability

Descriptors in radiology reports are often subjective and limited by the experience of the reporting radiologist and clinician.  For example, the burden of MS pathology affecting an individual's brain may be referred to a "scant, mild, moderate, moderately severe, numerous" etc. iQ-Solutions^TM uses an underlying database of >830 MS patients to provide an 'experiential' comparator against which individual patients can be consistently benchmarked.  In our cohort, this unique tool revealed inter- and intra-reporter inconsistencies in qualitative descriptors used in radiology reports.  Similarly, when brain volume loss is reported it is necessarily qualitative and severity is rarely described by radiologists.  Some quantitative tools, including iQ-Solutions^TM  provide comparison (using centiles)  of brain volume with an underlying healthy control cohort, but this provides little clinical context.  For example, when a radiologist or neurologist reviews a brain  MRI scan of a patient with MS, they compare this, albeit subconsciously, to the imaging of other patients with MS of similar age and disease duration  that they have seen over the course of their career, rather than to a hypothetical scan of an age matched healthy person.  Future research will determine the clinical utility of the brain volume experiental reference embedded within iQ-Solutions^TM. 

Integration with Clinical Workflows

Radiologists suffer high rates of burnout - the additional burden imposed by new apps, windows or other disruption of the clinical workflow imposed by some AI tools is a significant disincentive to their adoption.  Although our study was retrospective, iQ-Solutions^TM allowed us to mimick usual workflow by having annotated images returned to our research picture archiving and communications system (PACS) for viewing by the research team (Figure 2).   This was achieved using an integration hub (Torana^TM, Sydney Neuroimaging Analysis Centre, Sydney), which also automatically de-identified all images prior to their inclusion in the PACS, and added a unique (MSBase) identifier to the image meta-data to facilitate subsequent matching with the patient’s clinical data.  In clinical practice, annotated and re-identified images are returned to the clinical PACS where they are viewed alongside the original DICOM images to facilitate assisted reporting by the radiologist.  Importantly, our work describes a monitoring device for people with a known condition (MS), rather than a diagnostic tool.  Additionally, we show that some 10% of MRI scans of people with MS contain an incidental, unrelated abnormality of clinical significance, emphasising the importance of maintaining an expert "radiologist in the loop". 

Concluding Remarks

The precision management of multiple sclerosis, which targets No Evidence of (clinical or radiological) Disease Activity (NEDA), is now a realistic goal for many people with MS.   The integration of clinically validated AI tools into MRI monitoring algorithms will fill a crucial unmet need for MS radiologists and clinicians.   Workflow integration that retains an expert in the loop, combined with algorithms that add value but not burden, is the future of Radiology AI. 

The study is entitled “A Real-World Clinical Validation for AI-based MRI Monitoring in Multiple Sclerosis” published in npj Digital Medicine, 6, 196 (2023)"  Link is below:

https://rdcu.be/doW61