Tumor dynamic modeling has a long history, with many different types of models that have been developed over the past decades (reviews abound: e.g., here).  In the previous decades, the creation of tumor dynamic models have been driven by the ingenuity of human intelligence, using assumptions of mechanistic and/or empirical nature followed by the confirmation of the model fits. Much in the same way as how Newton stared into the night sky and pondered what equations might underlie the planetary motion:  

However, in the Digital Age we face the challenge of data growing along all of the following 3 dimensions: (1) the number of patients (e.g., from clinical trials and real-world data); (2) the dimensionality of data (e.g., genomics); (3) the number of longitudinal measurements (e.g., from digital devices). Not to mention the ever growing biomedical knowledge as reflected by the publications! 

It's time to ask: should we still model dynamical biomedical data similar to the way that Newton did? Alternatively, how can we evolve our modeling capabilities to best leverage the tremendous growth in both data granularity and volume?  We propose the AI-Partnered Modeling approach (see our prior work on neural-PK/PD modeling), whereby an encoder network is tasked with data abstraction and a decoder network based on neural-ODE is tasked with learning the underlying dynamical law:

In this work, we set up the foundations of Tumor Dynamic Neural-ODE (TDNODE). Key aspects in our model formulation include:

• Learning the dynamical law from data
• Interpreting encoder output as kinetic rate parameters
• Data augmentation to ensure robust, unbiased temporal predictions from early data 

Let's examine these aspects below.

Learning the dynamical law

Just like Newton assumed there's a universal law underlying the motion of planets, we assume that despite inter-patient variability, there is a certain dynamical law that hold across all patients in the clinical trial of interest.  Of course, just like planets differ in their initial positions and masses, so do patients in the clinical trial differ in their initial tumor sizes and kinetic rates. But there is an underlying (autonomous) dynamical system being learnt:

Note that other than the time evolution of z(t), there is no other explicit dependence on t in the right hand side of the equation! This is what makes it a time-independent dynamical law. Moreover, the dimensionality of p indicates how many parameters are being used to explain the observer variability in the tumor dynamic profiles. Therefore, by imposing the formulation as we have, we are making explicit assumptions on what type of model is being learned from the data.  This approach is different from simply applying, say, a Recurrent Neural Network onto the data, whereby we have impose little control on what kind of model is learned. Control on the formalism enhances interpretability. 

To be consistent with their interpretations, we do take care in feeding the encoders with portions of the tumor size data (SLD: sum-of-longest-diameter) so that their outputs correspond to initial condition and kinetic parameters (or, metrics) respectively:

How do we go about giving certain meanings to p? We look at that next. 

Imbuing meaning into parameters

The parameter encoder that we see above generates a patient embedding p from the SLD data. How do we imbue p with physical meaning, beyond simply a vector representation? If p is to have physical meaning, it should have certain units and scale according to those chosen units.  In this instance, we aim to interpret p as being kinetic parameters: i.e., physical quantities with the dimensions of 1/[time]. This gets to the notion of equivariance (eqn (2) of paper), resulting in a generalized homogeneity condition (eqn(3) of paper).

These mathematical derivations and the imposed data augmentation scheme effectively "teach" the neural network what it means for p to be kinetic parameters (as opposed to being dimensionless quantities). This is part of the ingredient of what makes the neural network "pharmacology-informed". 

Making unbiased predictions

A key aim of the (traditional) population modeling approach is to provide quantitative characterizations of the variability observed in the data. Note that, typically all of the longitudinal data is used. However, in situations whereby the longitudinal data is truncated (e.g., as available from an early data cut of a clinical trial - whereby some patients have long follow-up but patients that have only been enrolled recently and hence have short follow-up), there is no guarantee that the model's predictions of patients would be unbiased. While models are optimized and selected for their goodness-of-fit to the available data, that does not ensure predictivity at future time values. Indeed, examinations of these models have shown varying degrees of prediction bias (see Outlook).

In TDNODE, we have imposed a data augmentation scheme that focuses on making prediction for future times. I.e., we feed the early data into the encoder and ask the model to do as well as it can in predicting the unseen values. Here is an example whereby we feed in 16 weeks of longitudinal data and ask the model to minimize the discrepancy of predictions to the unseen values: 

As an analogy, it's akin to training a language model whereby it's given a few starting words and is asked for the next word prediction. 

Outlook

Why does this work matter? While existing tumor dynamic models are excellent for describing longitudinal clinical data, there is evidence indicating that they exhibit varying degrees of bias in predicting the future (unseen) tumor sizes. This may limit the applicability of existing tumor dynamic models in the setting of early decision making from clinical trial data.  In contrast, we have shown that as a result of the formulation of TDNODE, #AI has the potential to overcome this limitation. More on that in a future publication!

Another way that the TDNODE architecture represents an advancement is that it opens the door to leverage other advanced deep learning #AI models to aid cancer drug discovery and development. Eg, we have demonstrated how biological and pharmacological knowledge as represented in nettworks can be used in graph neural networks to make longitudinal tumor predictions. Much more along this direction is yet to come! 

To sum up: rather than just saving time and/or improving existing human constructed models, we believe that #AI can push on the frontier of what is possible to model. 

What would Newton do in this Age of AI?