Quantum probability for statisticians; some new ideas

It is vital to build bridges between various scientific communities. This paper is the end product on a long process, which includes: 1) several papers in physics journals, the latest one in J. Math. Phys. ; 2) a discussion paper in SJS; 3) a book to appear: On God, Complementarity, and Decisions.

The modern technological development would have been impossible without science. In a certain sense physics lies behind all sciences, and it is impossible to discuss modern physics without touching quantum mechanics in some way or other.

The great American physicist Richard Feynman said once: ’If somebody claims that he understands quantum mechanics, he lies.’ However, during the recent years essential new elements of understanding have appeared here.

Quantum mechanics is a fundamental physical theory that was developed in the beginning of the 20. century to explain a range of phenomena in the micro-world. This development is connected to important names like Bohr, Planck, Heisenberg, Schrödinger, and von Neumann.

One crucial point is that quantum mechanics is a formalism, a set of calculating rules for how one can predict the outcome of certain experiments. These calculating rules have had a large success; they have been used for everything from small elementary particles to complex chemical and biological systems, and in every case where this has been tested, the predictions have been 100% in agreement with the results of experiments.

In a book Helland (2021) and in a series of articles (Helland, 2022, 2024a-d) I have developed my own approach towards quantum mechanics. It relies on the notion of theoretical variables, assumed connected to an observer or to a group of communicating observers. The variables may be accessible or inaccessible, the first notion meaning that the variable in question may be measured arbitrarily accurately by the observer(s).

Using these notions, a set of 7 postulates are formulated, most of them quite obviously satisfied, except perhaps for two: 1) There is a basic inaccessible variable f such that all the accessible ones can be seen as functions of f. 2) The observer(s) has/have completely rational ideals. The point is that essential parts of the quantum formalism have been derived from these postulates.

Concretely, given the postulates and some symmetry assumptions, it has been proved in the articles above that a Hilbert space exists such that every accessible variable can be connected to a self-adjoint operator in this Hilbert space. This is a vital part of the quantum formalism.

In the finite-dimensional case, the symmetry assumptions may be dropped. In this case, the set of eigenvalues of an operator equals the set of possible values of the associated variable. An accessible variable is maximal in a concrete sense as such if and only if all eigenvalues of the associated operator are simple. Eigenvectors can be interpreted as concrete questions to nature together with sharp answers to these questions. The fact that all this can be derived from simple postulates, helps us really to understand essential parts of quantum theory.

In addition, quantum probabilities may be derived from these postulates. Here, the postulate 2) above is essential, together with assuming the validity of the statistical likelihood principle.

A general epistemic interpretation of quantum theory is natural from this approach: Quantum states (eigenvectors) can be interpreted as the knowledge that at each time is possessed by the observer or the communicating group of observers. This should be compared to statistical theory: The purpose of statistics is to find tools for achieving knowledge.

In fact, the link between statistics and quantum theory should be closer than this, and this is a main message of the present paper. First, it must be noted that the 7 postulates all can be interpreted in a way that is relevant also to macro-cosmos, not only the microscopical world.

This observation is consistent with the recent development of so-called quantum-like models, model that are relevant to psychology, in particular decision making, to finance, biology, and social sciences. A leading researcher connected to quantum-like modelling is the Swedish-Russian theoretical physicist Andrei Khrennikov; for a beginning of the theory, see Khrennikov (2010).

In the present paper it is argued from several points of view that quantum probabilities may play a role in statistical settings. This may be connected to the idea that more structure to the parameter space may be assumed, compared to classical statistical theory. Some such structure may for instance be related to model reduction under symmetry. A particular case here is the partial least squares model, see Helland (1990, 2025a), a model that has been generalized to envelope models in Cook et al. (2013).

Other arguments may be connected to recent developments in machine learning, a field that all statisticians claim is closely connected to statistics. There is a substantial literature on connections between machine learning and quantum mechanics, see for instance the review article by Dunjko and Briegel (2019). In a situation where such links exist, there should be an obvious task for statisticians to look for links between statistical theory and quantum theory. I see the present paper as just a beginning here.

A final class of arguments may be connected to the work by De Raedt and his collaborators; see for instance De Raedt et al. (2014). Here again, important parts of the quantum formalism are derived from reasonable assumptions. Important tools are the concept of Fisher information and the Bayesian-like probability theory of Jaynes (2003).

A recent summary of my work is given in the book Helland (2025b). Here, the 7 postulates are discussed in detail, and the consequences for statistics and quantum theory are made precise.  Also, many other consequences are discussed, consequences to religious faith, to sociology, political theory, and psychology, not least a theory of decisions. It is a sincere hope that all this in the long run may lead to better understanding across scientific borders.

References:

- Cook, R.D., Helland, I.S., and Su, Z. (2013). Envelopes and partial least squares regression. Journal of the Royal Statistical Society B 75 (5), 851-877.

- De Raedt, H., Katsnelson, M.J., and MIchelson, K. (2014). Quantum theory as the

most robust description of reproducible experiments. Annals of Physics 347, 45-73.

- Dunjko, V. and Briegel, H.J. (2019). Machine learning & artificial intelligence in the

quantum domain: a review of recent progress. Rep. Prog. Phys. 81, 074001.

- Helland, I.S. (1990). Partial least squares regression and statistical models. Scand. J.

Statist 17, 97-114.

- Helland, I.S. (2021). Epistemic Processes. A Basis for Statistics and Quantum Theory. Springer. (Revised version.)

- Helland, I.S. (2022). On reconstructing parts of quantum theory from two related maximal conceptual variables. International Journal of Theoretical Physics 61, 69.

- Helland, I.S, (2024a). An alternative foundation of quantum theory. Foundations of Physics 54, 3, arXiv: 2305.06727 [quant-ph].

- Helland, I.S. (2024b). On probabilities in quantum mechanics. arXiv: 2401.17717 [quant-ph]. APL Quantum 1, 036116. https://doi.org/10.1063/5.0218982.

- Helland, I.S. (2024c). A new approach toward the quantum foundation and some consequences. Academia Quantum 1. https://doi.org/AcadQuant7282.

- Helland, I.S. (2024d). Some mathematical issues regarding a new approach towards quantum foundation. Journal of Mathematical Physics.

- Helland, I.S. (2025a). On optimal linear prediction. arXiv: 2444412.19186 [math.ST]. To appear in Scandinavian Journal of Statistics.

- Helland, I.S. (2025b). On God, Complementarity, and Decisions. Consequences of a New Approach towards Quantum Foundation. Ethics International Press, UK.

- Jaynes, E.T. (2003). Probability Theory: The Logic of Science. Ed.: G.L. Bretthorst.

Cambridge University Press, Cambridge.

- Khrennikov, A. (2010). Ubiquitous Quantum Structure. From Psychology to Finance,

Springer.