Evolving beyond the scientific method: Since the Berlin Wall came down, there has been increasing investment in fundamental science, and an increasing number of scientific researchers throughout the industrialized world. Despite this, there have arguably been fewer notable scientific discoveries since the 1990s than in any decade from the 1940s-1970s. A plausible explanation is that the scientific method has picked most of the low-hanging fruits in science, such as natural laws that relate physical quantities via low-degree polynomials. That is, science has gotten harder. As a result, undiscovered scientific laws correspond to “higher hanging fruits,” which require new tools to pick. Discovering them would be useful, both to better understand the world and to contribute towards socio-economic growth. 

On the other hand, the increase in raw computing power and the efficiency of optimization algorithms has been one of the undeniable success stories of science in the past twenty-five years. Indeed, as noted in the full study, there has been an increase of six orders of magnitude in raw computing power between 1991 and 2015, and polynomial optimization has emerged as a practically tractable technology that leverages results from real algebraic geometry. 

The combination of stagnation in scientific discovery and acceleration in optimization suggests formulating scientific discovery as an optimization problem. Accordingly, we propose a new approach to scientific discovery that leverages recent advances in polynomial optimization. Namely, we formulate scientific discovery as a polynomial optimization problem that combines known laws of nature expressible as symbolic polynomials and numerical data that reflects experimental observations.

Our approach: Scientific discovery via polynomial optimization Given a set of background axioms, theorems, and laws expressible as a system of polynomial equalities and inequalities and observations from experimental data, we derive new laws representable as polynomial expressions that are either exactly or approximately consistent with existing laws and experimental data. We name our approach AI-Hilbert after David Hilbert, one of the first mathematicians to investigate the relationship between sum-of-squares polynomials, a set of polynomials that is easy to prove are non-negative, and non-negative polynomials (which can be quite hard to prove are non-negative); see this Quanta article for an accessible overview of sum-of-squares and non-negative polynomials.

Our approach provides an axiomatic derivation of the correctness of a discovered scientific law, conditional on the correctness of our background theory. The below diagram shows our approach and how it compares to data-driven and classical scientific discovery.

We contrast our approach with existing “classical” and “data-driven” approaches to scientific discovery below. Note that classical means new candidate laws are derived by manipulating existing scientific laws, while data-driven means that candidate laws are induced using experimental data. However, the problem of combining data and theory to derive scientific laws that are not discoverable using either data or theory alone, remains open.

In contrast, AI Hilbert combines data and theory to formulate hypotheses. Unlike conventional methods, insights into data and knowledge embedded in the theory collaboratively reduce the search space. These two components complement each other: theory compensates for noisy or sparse data, while data compensates for inconsistent or incomplete theory.

Results: As discussed in the full study, we illustrate our approach by axiomatically deriving some of the most frequently cited natural laws in the scientific literature, including Kepler's Third Law and Einstein's Relativistic Time Dilation Law, among other scientific discoveries. 

More details are available in the full study: nature.com/articles/s41467-024-50074-w

The full study does not require intimate knowledge of polynomial optimization to understand. However, if you want to try the approach for yourself and are not an expert in polynomial optimization, you may wish to learn more about polynomial optimization first. To do this, we recommend the excellent book “Semidefinite Optimization and Convex Algebraic Geometry” by Blekherman, Parrilo, and Thomas (2012). You might also want to check out our website https://ai-hilbert.github.io/, which contains Jupyter notebooks with the experiments we describe in the paper, in the Julia programming language.