There is much to like about the large-scale replication efforts reported by Ruggeri
et al. (2020) in Nature Human Behavior. First, there is the fact that their results cement the classic findings of Kahneman and Tversky (1979), providing a reassurance that is most welcome at a time when many phenomena long taken for granted turn out to be non-reproducible. Second, there is the impressive show of collaborative scientific work across different countries and continents.

But despite these merits, I cannot avoid finding the way that the authors frame their study (and the results coming from it) as somewhat controversial and counterproductive. Specifically, the authors appear to confound the unquestionable historical significance of Kahneman and Tversky’s 1979 paper with the theoretical significance of the items included in it. They argue that a failure to replicate Kahneman and Tversky’s results would have earth-shattering implications to behavioral science, whereas a successful replication would be tremendously valuable, as it would support Prospect Theory as an explanation for human behavior that could inform policy at different institutional levels. There are good reasons to disagree with both counts.

First, it is important the consider the specific role that these items played. They were used to create “choice paradoxes” that speak against Expected Utility Theory while sketching out the main tenets of Prospect Theory. That is as far as their usefulness goes. For instance, it is well known that they are grossly insufficient when it comes obtaining precise and reliable estimates of Prospect Theory parameters. Larger and more diverse sets of items are necessary, especially when attempting to compare individuals and/or groups (see Broomell & Bhatia, 2014). One of the reasons behind this limitation is the fact that the Prospect Theory’s general characterization of individuals as

1. being loss averse,
2. having a concave/convex value function for gains/losses, and
3. having a probability-weighting function that overweights small probabilities and
   underweights large ones,

is consistent with a large number of response patterns, some of which are at odds with the to-be-replicated response pattern that was originally reported by Kahneman and Tversky (e.g., Regenwetter & Robinson, 2017). These limitations are also reflected in the large body of empirical work studying in detail the different components of Prospect Theory (e.g., loss aversion, probability weighting), which use a variety of tailored experimental designs that largely differ from Kahneman and Tversky’s original work (e.g., Wu, Zhang, & Abdellaoui, 2005; for extensive reviews, see Wakker, 2010). If the case for Prospect Theory is to hinge on any specific body of empirical results, it shouldn’t be the set of items originally reported by Kahneman and Tversky but the many subsequent studies that more carefully dissect its components.

Moreover, it needs to be clear that the successful replication of Kahneman and Tversky’s choice paradoxes does not bestow Prospect Theory with any kind of privileged explanatory status, given that many other alternative accounts exist (e.g., Birnbaum, 2008; Loomes & Sugden, 1982; Lopes & Oden, 1999). To think otherwise amounts to affirming the consequent, a well-known reasoning fallacy that appears to be reinforced in cases in which successful replications are involved (for relevant discussions, see Regenwetter & Robinson, 2017; Rotello, Heit, & Dubé, 2015).

But more important than the mere existence of alternative accounts is the fact that in the last four decades, they have been used to derive and test “new choice paradoxes” that consistently reject Prospect Theory (for a review, see Birnbaum, 2008). For instance, consider the following pairs of lotteries:

A = ($110, .80; $44, .10; $40, .10)

B = ($110, .80; $96, .10; $10, .10)

and

A' = ($96, .80; $44, .10; $40, .10)

B' = ($96, .90; $10, .10)

According to Prospect Theory, people should prefer A over B if and only if they also prefer A' over B'. This prediction follows the fact that Prospect Theory still adheres to a weakened form of Expected Utility Theory’s independence axiom. However, the data speak loudly against this prediction, with a majority of participants preferring B over A along with a preference for A' over B' (Birnbaum, 2008). A number of paradoxes rejecting Prospect Theory were jointly replicated by Kellen, Steiner, Davis-Stober, and Pappas (2020).

At this point, I hope to have made it clear why the successful replication of Kahneman and Tversky’s original results has little bearing on the status of Prospect Theory. On one hand, the theory is consistent with a number of response patterns, some of which would be interpreted as failed replications. On the other, many different theories produce similar predictions, which prevents Prospect Theory from claiming any kind of special support. If the literature on decision-making under risk tells us anything, it is that the cases for and against Prospect Theory are to be made somewhere else. It's time to move on.

References

• Birnbaum, M. H. (2008). New paradoxes of risky decision making. Psychological Review, 115, 463–501.
• Broomell, S. B., & Bhatia, S. (2014). Parameter recovery for decision modeling using choice data. Decision, 1, 252–274.
• Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263–291.
• Kellen, D., Steiner, M. D., Davis-Stober, C. P., & Pappas, N. R. (2020). Modeling choice paradoxes under risk: From prospect theories to sampling-based accounts. Cognitive Psychology, 118, 101258.
• Loomes, G., & Sugden, R. (1982). Regret theory: An alternative theory of rational choice under uncertainty. The economic journal, 92, 805–824.
• Lopes, L. L., & Oden, G. C. (1999). The role of aspiration level in risky choice: A comparison of cumulative prospect theory and SP/A theory. Journal of Mathematical Psychology, 43, 286–313.
• Regenwetter, M., & Robinson, M. M. (2017). The construct–behavior gap in behavioral decision research: A challenge beyond replicability. Psychological Review, 12, 533–550.
• Rotello, C. M., Heit, E., & Dubé, C. (2015). When more data steer us wrong: Replications with the wrong dependent measure perpetuate erroneous conclusions. Psychonomic Bulletin & Review, 22, 944–954.
• Ruggeri, K., Alí, S., Berge, M., & et al. (2017). Replicating patterns of prospect theory for decision under risk. Nature Human Behavior.
• Wakker, P. P. (2010). Prospect theory: For risk and ambiguity. Cambridge, Mass., Cambridge University Press.
• Wu, G., Zhang, J., & Abdellaoui, M. (2005). Testing prospect theories using probability tradeoff consistency. Journal of Risk and Uncertainty, 30, 107–131.