Researchers live in fear of being “scooped.” Understandably so, as being “scooped” often means that hours, weeks, months, or even years of your hard work and scholarship have seemingly been wasted. So, imagine that after you've submitted your first paper, you receive an email from the editors of Nature Human Behavior telling you that another research group had submitted a similar paper at the same time, analyzing the same dataset, with more or less the same results. Furthermore, imagine you had submitted those manuscripts within three days of each other. And, even though your methods were different, both teams had found almost identical results.
I don’t have to imagine this, because this is exactly what happened during the publication process of our new paper, “Negativity Increases Online News Consumption.” My team (made up of myself, Dr. Philip Pärnamets, and Dr. Jay Van Bavel) and another team (Dr. Stefan Feuerriegel, Kaoru Schwarzenegger and Dr. Nicolas Pröllochs) had unknowingly spent months working on similar projects, both using the same incredible dataset from the Upworthy Research Archive (Matias et al., 2021)
The Upworthy Research Archive is a dataset containing years of real-life news consumption data from Upworthy.com. Upworthy.com used A/B testing from 2012-2015 to figure out what features of headlines would lead the most people to click on an article. For each article they published, they tested up to nine headlines via random assignment to Upworthy readers. So, for a story about the Supreme Court striking down Prop 8, which prohibited gay marriage in California, two possible headlines were “Wow, Supreme Court Have Made Millions Of Us Very Very Happy,” and “We’ll Look Back at This In 10 Years And Be Embarrassed As Hell it Even Existed.”
Upworthy was one of the originators of “Click-Bait,” which is an internet-news phenomenon where news outlets post attention grabbing headlines in order to garner more clicks on their websites. For online news sources, clicks are effectively currency, so Upworthy wanted to find out how to optimize their headlines for maximum clicks. All in all, Upworthy tested over 105,000 headlines and recorded over 518 million unique visits to their website. Furthermore, because story content was controlled for each headline package, and headlines were randomly assigned, this allows researchers to analyze a massive and applied dataset that is also causal.
Because the dataset is public, two research teams had decided to work on the same questions using the same data. And, we had both submitted to Nature Human Behaviour. Upon finding this out, I was convinced that I was about to experience my first “scoop” on my very first paper. However, Nature Human Behaviour offered an unusual but ultimately excellent suggestion – instead of pitting our two teams of researchers against each other as competing papers (likely resulting in one team getting scooped), the editors saw an opportunity to raise the confidence in these converging results. The editors therefore proposed an interdisciplinary collaboration between our two teams—our team coming from a background in social & cognitive psychology, the other team from the field of computer science. They asked us to work together to blend our manuscripts into one unified work. We eagerly agreed.
With that, we each met our new, unexpected collaborators. Our teams approached the theoretical questions of the projects from different perspectives, and due to the diversity of perspectives on our new team, our proposal grew theoretically and technically much stronger after joining together. Furthermore, an upside of the pandemic was increased ability to work with geographically diverse teams. Even being on different continents was not a detriment to our blended team.
Another feature of this process that was unique was that both teams submitted our initial manuscripts as Stage 1 Registered Reports. We combined teams and manuscripts to submit an improved Stage 1 proposal, which was ultimately accepted. We then ran our analyses on previously withheld data to confirm our hypotheses.
Across our initial analyses, our joint Stage 1 pilot analyses, and our Stage 2 confirmatory analyses, we found that negative language in news headlines increased the likelihood that a given headline would be clicked on. We also find that positive language decreases the likelihood of a headline being clicked on. After working together to refine our analysis plan, we also found that sadness seemed to be the main emotion driving clicking behavior, while we found no effect of anger, which is often the emotion associated with online virality. Notably, however, we do have a different dependent variable in our study -- we look at consumption, which is a private behavior, while many prior studies look at share rates, which is a public behavior.
In addition to the findings in our manuscript, I hope we can also learn from this unorthodox process. Researchers can become rather territorial about their work. However, this is due to the nature of our publication process, not the scientific method itself. Indeed, finding out that an independent group of researchers has found exactly the same effects as you is scientifically ideal – one could think of this type of phenomenon as a simultaneous replication. Indeed, it was deeply reassuring to me that, even though both teams initially used different methods and different dictionaries, we got meaningfully identical results.
It is also unusual that two teams who did not know each other would converge on the same topic. But, as psychology becomes increasingly integrated into biology, computer science, sociology and political science, there may be more occurrences like this. Working across disciplines offers a host of benefits, including challenging discipline-specific assumptions and writing more integrated theoretical backgrounds. Working with experts in computer science greatly increased my technical literacy, which is invaluable as an early career scientist.
I believe this experience shows a possible model for future collaborations. I think it would overwhelmingly be a good thing for science if people were encouraged to team up to share skills and insights, rather than race to produce out of fear of being "scooped".