Well, so much for the myth of the disinterested scientist.

The idealized, data-oriented view on research seeking absolute foundations of knowledge pursued by “objective” scientists has been falsified by almost all major scientific discoveries, and has been denounced by Popper (2005). The highly structured format of the research paper, presenting an idealized chain of events, from hypotheses, through the evidence, to the conclusions, gives rise to a "fraudulent" misrepresentation of the actual thought processes that led to the work (Medawar, 1964), thus perpetuating the myth of scientists doggedly abiding by the linear method. “There is no such thing as unprejudiced observation.” (p.42, Medawar, 1964) A researcher pretending to be disinterested (i.e., “objective”) would be less than transparent and could even be seen to be hypocritical and lacking sufficient intellectual honesty. New discoveries usually result from an idiosyncratic, sometimes erratic, highly nonlinear quest for a better understanding in uncharted territories, full of wrong turns, failures, and a rare success (cf, Firestein, 2015; Lehrer, 2009). Reformers sought to “linearize” this messy research process by implementing strict methodological guidelines and getting rid of theoretical “chatter”. The linear conception hinges on illusory retrospective reconstructions of the research process as the privileged path to "the truth," and on ignoring the probably infinite plurality of viable options faced prior to it. (p.223; Phaf, 2024)

The involvement in this retraction (Protzko et al., 2024) of prominent reformers who advocate strict normative methodologies demonstrates, in my view, the utter impossibility of pre-emptively regulating such an immensely complex endeavor as (psychological) science.

These methodologies may indeed even be harmful to successful science.

Normative methodologies are unable to differentiate between positive and negative deviants either beforehand (i.e., through pre-registration) or afterward (i.e., through replication). The narrow limitation to data (i.e., statistical testing) and the eschewing of pre- and post-experimental theorizing impede rather than strengthen the tinkering necessary for the successful evolution of psychology. Many reformers argued that data should be shielded from theory to prevent biased data processing (Munafò et al., 2017). In their quest for humility, Hoekstra and Vazire (2021) even suggested that researchers should remain agnostic about which interpretation is most likely to be valid for their data. The reformers oppose "hypothesizing after the results are known" (HARKing; Kerr, 1998), thereby conflating insubstantive statistical hypotheses with substantive theoretical hypotheses, as demonstrated by their many failures to replicate statistically in the absence of any theoretical elaboration. Firestein (2015) argues that failure is the basis for successful science. The reformers focus on data failures, which they attempt to prevent at the outset. Pure data, however, can never fail, only hypotheses can fail, and exclusively in competition with other hypotheses. Furthermore, by prohibiting HARKing, researchers are denied the chance to learn from failure. As theory is greatly underdetermined, and certainly not dictated, by empirical data, imaginative, substantive insights are invariably required (i.e., theory-laden guesses; cf. Deutsch, 2011). Banning HARKing means that hypotheses can only be rejected but not replaced by more useful ones, resulting in "a race to the bottom." (p.227; Phaf, 2024)

For a successful evolution of (psychological) science, a reorientation from a primacy of data to a primacy of theory is a first requirement.

Psychological science must not remain “a tiny, frozen island of explanation in an ocean of incomprehensibility” (p.446; Deutsch, 2011). Only radical innovation, sometimes breaking with current methodological norms, will help it escape from local fitness maxima to reach even higher utility levels, without ever arriving at a nonexistent global maximum. To promote productive science, the susceptibility to, rather than the prevalence of, deviant findings and ideas must be enhanced. Positive deviants rarely, if ever, arise in a theoretical void, and will emerge more readily if the literature is filled with publications that articulate their theoretical hypotheses. The most impressive spin-off from psychological research today, for instance, is due more to theoretical than experimental work. The current “AI revolution” emerged from the connectionist modeling of learning systems, and is loosely built on results from experimental psychology (cf. LeCun et al., 2015). These learning systems can, in turn, accelerate successful science by enabling automated semantic analyses that can discover latent hypotheses in the overwhelming volume of publications, and may ultimately suggest disruptive innovations (cf. Sourati & Evans, 2023; Weeber et al., 2001). (p.228; Phaf, 2024)

References

Deutsch, D. (2011). The beginning of infinity. Penguin Books.

Firestein, S. (2015). Failure: Why science is so successful. Oxford University Press.

Hoekstra, R., & Vazire, S. (2021). Aspiring to greater intellectual humility in science. Nature Human Behaviour, 5(12), 1602-1607.  https://doi.org/10.1038/s41562-021-01203-8

Kerr, N.L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196-217. https://doi.org/10.1207/s15327957pspr0203_4

LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444. https://doi.org/10.1038/nature14539

Lehrer, J. (2009, December 21). Accept Defeat: The Neuroscience of Screwing Up. http://www.wired.com/2009/12/fail_accept_defeat/2/

Medawar, P.B. (1964, August 1). Is the scientific paper fraudulent? Saturday Review, 42-43. https://www.unz.com/print/SaturdayRev-1964aug01-00042/

Phaf, R.H. (2024). Positive deviance underlies successful science: Normative methodologies risk throwing out the baby with the bathwater. Review of General Psychology, 28(3), 219-236. https://doi.org/10.1177/10892680241235120

Popper, K.R. (2005). The logic of scientific discovery. Routledge. (Original work published in German, 1935). https://doi.org/10.4324/9780203994627

Protzko, J., Krosnick, J., Nelson, L., Nosek, B.A., Axt, J., Berent, M., ... & Schooler, J.W. (2024). Retraction Note: High replicability of newly discovered social-behavioural findings is achievable. Nature Human Behaviour. https://doi.org /10.1038/s41562-024-01997-3

Sourati, J., & Evans, J.A. (2023). Accelerating science with human-aware artificial intelligence. Nature Human Behaviour, 1-15. https://doi.org/10.1038/s41562-023-01648-z

Weeber, M., Klein, H., de Jong‐van den Berg, L.T., & Vos, R. (2001). Using concepts in literature‐based discovery: Simulating Swanson's Raynaud–fish oil and migraine–magnesium discoveries. Journal of the American Society for Information Science and Technology, 52(7), 548-557. https://doi.org/10.1002/asi.1104