Artificial intelligence (AI) tools are increasingly used to make important decisions about individuals in sensitive fields like healthcare, finance, and security. While these AI models might be accurate when looking on average at a large group, they can be highly uncertain about the outcomes for specific individuals or smaller groups. For high-stakes situations, it's crucial for AI decision-support systems to provide a rigorous assessment of uncertainty for each individual's prediction, so as to provide a personalised assurance that the AI system may be trusted and is fit for purpose. One promising approach discussed in the paper is Conformal Prediction that can use a calibration step to ensure that the uncertainty in AI predictions fall within guaranteed bounds at a level of significance set by the user.
Personalised uncertainty quantification allows decision-makers to understand the confidence level of AI recommendations and weigh them against other considerations or information to make more informed decisions. The Alan Turing Institute (UK's national institute for AI) hosted a series of workshops organised by the Turing-Roche Strategic Partnership where a multidisciplinary cohort of data scientists, policy makers and industry leaders discussed the future of predictive modelling and published a perspective paper in Nature Machine Intelligence that outlines eight forward looking grand challenges in the roadmap towards achieving personalised uncertainty quantification.
- Individualised predictions: developing tools that provide uncertainty guarantees tailored to each specific individual, not just on average.
- Multiscale modelling: assessing uncertainty in models that combine data from different levels of detail or resolution, like microscopic and macroscopic information.
- Multimodal AI: creating ways to assess uncertainty when combining diverse types of data, such as genetic information and medical images, and adapting to new data types as they emerge.
- Explainable design: using personalized uncertainty assessments to help explain why an AI made a particular decision, thereby increasing trust.
- Monitoring models: using personalised uncertainty quantification to track how well AI models perform over time and to detect when changes in data might cause them to degrade.
- Missing data: addressing how missing data, especially if it's systematically absent for certain groups, affects the uncertainty of predictions for individuals.
- Equitable decisions: ensuring that personalized uncertainty assessments operate fairly across different population groups (e.g., based on age, ethnicity, or gender) to prevent biased outcomes.
- Generative AI: quantifying uncertainty in AI-generated "hallucinations" (plausible but factually incorrect outputs).
Personalised uncertainty quantification also increases the trust of individuals to better understand AI-driven recommendations, and act on them appropriately. This dovetails into the concept of “justified trust” and assurance that an AI system is “fit for purpose” in a high-risk application and connects to the call for AI assurance by the UK Govt in its recent white paper. One example could be use of trustworthy AI to complement clinical decision making in multidisciplinary team meetings (MDTs) and thus improve patient outcome, as outlined in this recent Nature comment article by the Turing-Roche team. The Turing-Roche YouTube channel also has a public oriented video summary of this project and related ones.