Causality in predictive analytics

How the story behind the data can tell us about opportunities and pitfalls in machine learning for medical imaging
Published in Protocols & Methods
Causality in predictive analytics

Share this post

Choose a social network to share with, or copy the shortened URL to share elsewhere

This is a representation of how your post may appear on social media. The actual post will vary between social networks

In the summer of 2018, the idea of applying causal reasoning to medical image analysis came to be a major focus for us, spurred on by Judea Pearl and Dana Mackenzie's 'The Book of Why' [1]. Pearl and Mackenzie make a compelling case for how causality should play a central role in modern data science and predictive analytics. We started to ask ourselves how a causal perspective could augment our own research, at the intersection of machine learning and medical imaging. The idea that causal reasoning could help us to better understand some of the key challenges we face when developing predictive models for clinical applications captured our imaginations.

Figure 1: Cartoon illustrations of key challenges in machine learning for medical imaging.

At that time, deep learning had already taken the field of medical image analysis by storm. Impressive performance had been achieved for automated detection and classification of disease, and new state-of-the-art results were reported on a regular basis for challenging tasks like automatic segmentation of diverse pathological structures (such as brain tumours). The availability of well-curated and reasonably large, public datasets has contributed much to the success of these methods. Annual challenges, held to test and evaluate the latest algorithms for various tasks, have consistently been dominated by deep learning algorithms, often showing performance on par with human experts. However, only a few algorithms have successfully made the transition from the lab to the clinic. Two possible reasons for this failure have been discussed extensively in the literature and in our Perspective.

First, it quickly emerged that many algorithms are not sufficiently robust, meaning their performance may drop considerably when tested on data that differs from the datasets used during development. In medical imaging, this often occurs when data is acquired from different machines or clinical sites. This problem of generalisability is well studied in machine learning, and, while strategies exist to mitigate it when the cause of the dataset shift is known, there has been no systematic characterization of this issue in medical imaging.

Second, whenever a new clinical application is explored, one starts from scratch in terms of data collection and annotation. Therefore, researchers in medical image analysis are constantly faced with a shortage of high-quality annotated data to train deep-learning algorithms. To overcome this challenge, one may opt to use data augmentation techniques or even try to leverage entirely unlabelled images for improving predictive models. Yet, these techniques are not well understood and it often remains unclear in which cases they may prove beneficial.

Our Perspective reveals that the answers to some of those questions are hidden in the stories behind the data. Here, the language of causality allows us to meticulously describe the processes that generated the data we use to build predictive models. Causal reasoning both allows and requires us to clearly communicate our assumptions about the world, and can help identify potential pitfalls such as dataset shift and sample selection bias, or even whether certain learning strategies should be considered at all. But, this is really only the first step toward climbing the ladder of causation [2].

Figure 2: Visual results of counterfactual image generation with a simplified structural causal model relating age (a) and biological sex (s) with brain volume (b) and ventricle volume (v). The top row shows from left to right the original input image and counterfactuals generated with our deep learning model corresponding to different interventions. For more details see [4].

The next natural question is whether we can directly integrate causal assumptions about the world into our models, which gives rise to the field of causal representation learning [3]. Our own early attempts to integrate structural causal models into deep learning have led to some promising results, showing it is possible to generate plausible counterfactual images [4] for questions such as 'What would this patient's brain look like if they were 10 years older?' (see Fig. 2). Being able to answer counterfactual 'what if' questions may well hold the key to unlock the full potential of AI in predictive analytics and healthcare applications—an exciting area of research for the decades to come.


[1] Judea Pearl and Dana Mackenzie (2018). The Book Of Why: The New Science Of Cause And Effect. Basic Books, New York.

[2] Judea Pearl (2009). Causality: Models, Reasoning, and Inference (2nd ed.). Cambridge University Press.

[3] Bernhard Schölkopf (2019). Causality for Machine Learning. Preprint at

[4] Nick Pawlowski, Daniel C. Castro, and Ben Glocker (2020). Deep Structural Causal Models for Tractable Counterfactual Inference. Preprint at Demo and code at

Please sign in or register for FREE

If you are a registered user on Research Communities by Springer Nature, please sign in

Subscribe to the Topic

Biological Techniques
Life Sciences > Biological Sciences > Biological Techniques

Related Collections

With collections, you can get published faster and increase your visibility.

Applied Sciences

This collection highlights research and commentary in applied science. The range of topics is large, spanning all scientific disciplines, with the unifying factor being the goal to turn scientific knowledge into positive benefits for society.

Publishing Model: Open Access

Deadline: Ongoing