Behind the scenes of a Nature Climate Change paper

Eight years ago, in an unexpected way, the idea for this project emerged. Here, I will take you through the journey of how an idea turned into a research project and the process to materialize it. A tale that involves unexpected interest groups, the CFTC, many obstacles, and a surprising ending.
Published in Social Sciences
Behind the scenes of a Nature Climate Change paper
Like

Share this post

Choose a social network to share with, or copy the shortened URL to share elsewhere

This is a representation of how your post may appear on social media. The actual post will vary between social networks

I am a neuroscientist. I study the human brain and focus on “Consciousness”. I am not a climate scientist. In fact, when this project started, my knowledge of climate science was limited to what one can glean from a lay reading of the news media. That was eight years ago when I embarked on a journey that culminated in today’s publication of two works: a research paper  (https://www.nature.com/articles/s41558-023-01679-4) and a policy brief (https://www.nature.com/articles/s41558-023-01677-6) in Nature Climate Change.

Like many academic journeys, it started at a conference. Although this one was not an academic one. Rather, it was a “popular” conference. In 2015, Northwestern University's Board of Trustees invited 3 scientists whose work they deemed “remarkable” to speak about the findings. I was the second presenter that day. I spoke about consciousness and the progress we were making toward answering the biggest question in neuroscience. I ended by saying that making progress with this research might help us expand our ability to overcome other major challenges in the world and named a few. One of those was Climate Change.

My view was that some of the climate challenges we face are the result of a flawed brain.

***

Our brain has a hard time projecting itself onto entities outside of ourselves (what is it like to be a bat?), it is challenged with thinking of extremely small things (the size of a proton compared to an electron), on things that are not deterministic (what does it mean that the quantum world works in probabilities?), of extremely big things (what does it mean that the universe is infinite?), or things that span wide spatial or temporal horizons (what would it feel like to be a retired 70-years old man, when I am only in my mid-20s?).

The temporal horizon challenge is particularly relevant to climate change. When we clap our hands and hear the sound of the clap, it is easy for our brain to connect the cause (clap) and effect (sound). When things start spanning wider time windows our brain fails to connect the dots. Thunder and lightning happening seconds apart sometimes confuse kids who find it hard to understand that the two events were initiated simultaneously. Now imagine that we would clap our hands on Tuesday and the sounds would be heard 80 years later.... How easy would it be for our brain to see the cause-effect relationship here?

If the trash that we recycle would immediately cause a hurricane to become smaller, we would have an easier time handling climate attribution. Unfortunately, many climate events are far apart temporally and non-linear in their nature. To say that our brain is challenged with seeing the connection between those events is an understatement.

In fact, if we were to design an “ideal” challenge for the human brain, climate change would be it. Anything we do as individuals is a drop in the bucket; the actions we take today may only have an impact in the future (sometimes decades in the future); there is a great uncertainty (partially, fueled by opposing interests) about various solutions; the outcomes include multiple variables; we see the cost (i.e., taxes to solve the problem) but not the benefit (i.e., if we work hard to mitigate climate change all that will see is that a lot of bad things are not happening, but not that something is happening) and, “the icing on the cake”, the damage we cause in one place (say, the U.S.) is likely to hurt others far away (i.e., people in Bangladesh) first.

So why should I spend money, time, or energy, fixing a problem that has all those characteristics when at the same time there are immediate things I can worry about here and now (mortgage, tenure, my dog being sick, etc.)?

While the climate catastrophes are clearly more rapid, closer to home, and much more devastating than predicted - repeatedly beating the most horrific estimates - seemingly, our brains have not adapted to thinking about them properly. Our biological neural underpinning still believes we are in the savannah with a lot smaller challenges The world has advanced technologically since, but our brains did not. Put differently, we have a Paleolithic brain handling godlike challenges, with medieval institutions.

***

I ended my talk back then suggesting that technological advances in neuroscience may be used to tackle big societal challenges like financial literacy (making young people save for retirement) or climate change. This was not hyperbole. One of the projects I was working on at the time had to do with dream manipulation and the idea that we can make the brain see cause and effect differently if they happen during a dream state when it accepts temporal shifts more readily.

Following my talk, Professor Malcolm MacIver took the stand. Malcolm is a professor of engineering. He spoke about our senses. Specifically, how senses were developed to match the reality of our existence. Fish cannot see far away given the optics of light in water. This puts the fish at risk of missing a predator when it approaches from afar. Therefore, the fish had to develop ways to sense predators using other means (along with the ability to respond fast). Zebrafish, Malcolm's “pet animal”, senses perturbations in the electromagnetic field of water when a predator approaches, way before it can “see” the predator. This way the fish can anticipate future trouble ahead of time. If only there was a way to use engineering to help our brain connect future events to current needs maybe there would be a way to harness technology to mitigate some of the risks that humanity faces, Malcolm suggested at the end of his talks. He referenced my talk's ending and said, on stage, that maybe the two approaches - brain and technology - could be combined to tackle one such looming risk.

That is how it started.

When the meeting was over we drove together to campus. We talked. We talked more. We had numerous meetings where we attempted to think of ways to build devices that will help the brain consider future events and make better predictions about them. But we could not figure it out. The solution came from an entirely different discipline – neither neuroscience nor engineering. Emphasizing, yet again, the importance of interdisciplinary research and working outside traditional lanes, the solution came from research in… business.

***

At Northwestern, I held a dual appointment. In addition to being a professor of neuroscience, I was also a professor of business, teaching at the Kellogg School of Management. It was my business school professor hat that culminated in the answer: What if we can use future markets to make give people immediate feedback on distant outcomes?

The idea was inspired by the work of a colleague of mine, Professor Hal Hershfield at UCLA. He attempted to help young undergraduate students think differently of retirement savings, using technology. In Hal's case, he brought the students to the lab and exposed them to computer-generated images of themselves as older people. Following the visceral visualization of yourself as older participants took a different perspective on savings in a following survey

We set out to apply similar thinking in the context of climate change: use a technological instrument to bring the future closer. The instrument we landed on was a “prediction market”.

***

Prediction markets are a fancy name for “bets about the future”. You say that this horse will win the race. I say that it won't. We both put money in the pot. The race takes place. One of us is right and they take the other person's wager. We both made a prediction on the future with money on the line. It becomes a market if we can also trade predictions. If you can sell your position on the horse to someone else, say halfway through the race when you see that your horse is 200 meters ahead of everyone else and your chances of winning are high. The buyer sets the new price and takes your position in the bet.

This is, in a nutshell, a prediction market. People make predictions on the future, take positions on some outcomes, trade their positions with a value that is alternating with new information, and ultimately earn/lose money when the bets settle.

Prediction markets are an extremely powerful tool for estimating future outcomes, across various domains. One of the more popular domains is politics. Prediction markets along the lines of "who will win the U.S. Presidential election anticipated Joe Biden's win over Donald Trump months before Biden was even a nominee. The markets predicted accurately numerous mayoral and gubernatorial races, the Brexit outcome, and many others. The key is that unlike polling, where people typically state their personal wishes, in a prediction market people actually predict what they believe will actually happen. They do so because they stand to gain money from being accurate. For example, I might really want the Oakland Raiders to win the Super Bowl but recognize that they are very unlikely to do so. I then might go cheer for the Raiders but actually put my money on their opponents when it comes to predicting the winner.

The key power of prediction markets stems from the fact that they force us to make decisions in the present about future outcomes, and update those decisions (by trading positions) with new information. If we take a position that the global average temperatures will not increase by 2 degrees over the coming decade (say, because we believe that the climate science predictions are wrong), we might try to sell our position and recoup some of our money if in 5 years the temperatures already reached 1-degree increase. We do not have to wait for the bet's expiration date in order to use new information in our financial trading.

This logic allows for the use of prediction markets to help people foresee future events.

In the context of climate change, we needed to set bets on future climate events and let people make predictions that will force them to project future climate outcomes to current decisions: will the number of wildfires in California increase above the average of the last decade? will the average hurricane sizes increase? will the water levels in Miami reach all-time highs? will the average Air Quality Index be higher than ever before? etc.

Setting climate bets that afford people to make smart, informed predictions, is not trivial. It required identifying realistic topics that are truly controversial. For example, if we were to set a bet that was too extreme (say, that the average temperatures will increase by 50 degrees in the next decade) then the likelihood of anyone taking opposite positions would be low (also, in the particular example, if it were true, there won't be anyone to claim the prize money...). Additionally, bets needed to have relevant sources that would determine the outcome (i.e., if we were to determine who won the 2020 U.S. Presidential election based on Breitbart’s reporting we would have had likely disputes over the actual results). Finally, we had to think about bets that could be resolved with some data rather than pure intuition. If a bet, say, asked whether it will rain tomorrow, then it won’t really be a climate bet necessarily and might be less data-driven and more variable and chaotic.

In short, our idea faced some challenges when we actually considered its implementation. We also were not sure that the short timescales of a typical lab study (months) would actually manifest any remarkable climate events. Mostly, we were concerned that if we randomly picked a time window within which we’d run the study no notable climate events that we could be used for bets (i.e., extreme heat, or unusual wildfires) would occur. Unfortunately (for the world), we quickly learned that any period of time we chose had an abundance of those extreme climate events.

***

Armed with the idea of a prediction market as a tool for bringing the future to the present, we now had to design a study that will actually prove to us that their future projections indeed help the brain see things differently. We needed a way to measure the manifested change in thinking. The brilliant way to test this shift came from Professor Sandra Matz who joined us at this part of the journey. To show that climate prediction markets help the brain think differently about the future we want to show that there is a change in people’s perspective on climate change after spending time engaging with the prediction market. We needed a before/after design where we could demonstrate shifts in people’s views. And we needed to know in what ways people’s minds will change.

We thought of two ways by which people can change.

First, they might know more and engage more with climate topics. Simply, if you buy a stock in the financial market (say, Apple stock), you are more likely to care about Apple, maybe pay more attention to news about them, or even buy their products and advocate for them. You, literally, have stock in their success. Participating in the climate prediction market, we thought, would make people behave the same when thinking about climate change. They will learn more and think about it more if they have money on the line. Potentially, we thought, they may learn things by witnessing the strengths of hurricanes in the Caribbean, the number of earthquakes in Turkey, or the Atmospheric River in San Francisco. This would be amplified if their participation in the climate market will yield financial gains for them.

Second, maybe spending time making predictions about climate and seeing that, overwhelmingly, the outcomes align with the models suggested by scientists, will strengthen people's alignment with the reality of climate change. It might even drive them - outside of the financial realm in our study - to support climate action (i.e., commit to spending money, voting, or advocating for climate issues).

We would need to measure knowledge, support for climate action, and overall views about climate science after people participate in a prediction market.

***

Once we knew what we wanted to measure, we created our first study. In this study we had participants fill out a long (some say, tedious) survey where we probed their climate knowledge, their views about climate science (is it anthropogenic, is it risky, ...), and their support for action.

To strengthen the effect, we also took one bolder move. We decided to not just recruit participants regularly (i.e., through online panels) but rather turn to places where we can find “extreme” participants. Specifically, since we wanted to see if we can truly change people's views in a way that makes s difference, we sought to target climate deniers. Through Reddit's “Climate Change Skeptics” group we recruited participants who were least likely to change their views about climate change: people whose identity and ideology are ingrained in an opposition to climate science.

We balanced those participants with a subset of climate believers, which we recruited using an online pane and started our study.

Participants in the climate market spent weeks making bets on climate outcomes and winning/losing money based on their accuracy.

To ensure that people can trade positions in real time we had to build our own online, robust, climate market. The market had to serve trades all day long, it had to support heavy traffic when new bets were launched, it required “customer support” (when people forgot their password or claimed they made trades that did not register), and an overall level of logistical management that was far more demanding than the typical lab study (the nature of real-time massive field study). Given that our expertise was primarily in sterile lab studies, these operational demands were challenging. With the help of numerous Research Assistants, Silicon Valley programmers who graciously donated their time to help build the cloud support, and by devoting numerous sleepless nights to materializing the climate site, we managed to pull the first study.

Nonetheless, when we came to analyze the results, it seemed like we failed to observe the predicted effect. There was no notable change in people's views, knowledge, or support for climate action when we compared participants' answers before/after the weeks-long participation in the climate prediction market. We also did not see a big difference (albeit the small difference was significant) between participants' views on climate issues if they were part of the climate prediction compared to control participants who sat idle for the entire prediction period. A failure?

Professor Matz saved the day here. Looking at the interaction between winning bets and climate views was the key. If you just play in the market for a while - nothing happens. But, if you are consistently winning your bets (meaning, you are actually making accurate predictions) you show a significant change in your views about climate change. The change is also in the expected direction: towards more alignment with climate science.

That is when four critical things happened that changed the course of our project:

  1. We submitted the paper to Nature Climate Change and received positive reviews with one critical request: “Redo the study with a different control. Instead of having a control group that sits idle for a month, we would like to have a control group that actually participates in a non-climate prediction market”. If you can show that they do not change their views and knowledge, you will be certain that the climate prediction mark drove the shift in views.
  2. We realized that if we were to redo the study, we should get help from a corporate partner who would be more equipped to handle the operational challenges of running a real-time study (also, those studies are very expensive since participants are paid proportional to the accuracy of their predictions).
  3. We realized that we handicapped ourselves in the first study by using a polarized group. While those are the most interesting individuals with respect to “changing views” (climate deniers are the target of numerous perspective change efforts) they are also the hardest to move. In business school, we always tell MBA students that a marketing effort aimed at those who hate you is much harder than one that targets people who already see some value in your offering (imagine, for example, Samsung trying to sell phones only to avid Apple users). Accordingly, trying to shift the deniers would be much harder than focusing on those who are in the middle.
  4. We became more ambitious in our study goals. If prediction markets are useful in changing people's views about a polarizing topic such as climate change, maybe they can be used in other domains of controversy. Maybe we can even solicit the help of the government and create a global prediction market where people can trade positions on opinions and learn about the truths through this financial instrument. We also saw the potential of using the prediction market for policy-making, public opinion polling, or as a tool for increased science alignment and knowledge.

Armed with those thoughts, we set the course for the next phase of this project. The time was March 2020. Specifically, with those ideas in mind, we started designing Study 2.

***

Step one: reaching out to prominent organizations that run prediction markets, at scale, to solicit their help as partners. Turns out, this was a dead end. Not a quick one, but a dead end nonetheless. We spoke to companies like PredictIt (www.predictit.org) that ran, at the time, the biggest prediction market website, and asked for their help. They had us talk to lawyers, consultants, lobbyists, and technical people. In the end, it became clear that they are bound by various regulations that prevent them from doing what we needed. We spoke to Kalshi (www.kalshi.com), an emerging website that entered the market around that time. Again, we spoke to numerous people from their team only to learn that their website falls short, technically, of our capacity demands (and also, that their development speed is too slow for our needs; we always thought that academia is much slower than industry... turns out this is not always the case). Finally, we spoke to the CFTC, which is the regulatory body in charge of all consumer betting and futures.

Prof. Cerf speaking with the CFTC Chainr
Prof. Cerf speaking with the CFTC chair, Rostin Behnam

Climbing up the ranks of the CFTC we ended up speaking to the then-acting chair of the organization, Rostin Behnam, who seemed excited and eager to facilitate our initiative to make the federal climate prediction market a reality. We were thrilled. But... then politics happened. “Acting” chair means you are appointed by the ruling President. at the time this was Trump, who just recently lost the election to Joe Biden. Chairman Behnam, transparently, said that the CFTC was in a state of transition. He himself was not sure if he would keep his position and what the new Administration's position on climate would be. It did not help that the world also was enduring its biggest pandemic in a century. In short, we will have to wait. We did.

One year.

After a year, we decided we cannot wait any longer. We would have to go back to the drawing board and find a way to run a big study, at scale, with hundreds of participants, and two parallel prediction markets (one about climate, and a control one, on entertainment). But... we already learned that doing this at scale is very difficult when people need to trade bets in real-time. We looked for a solution that would allow for controlling the participation dynamic.

After multiple iterations, we landed on a solution. Instead of a market where people trade options (meaning, trade with each other) we will take the counter positions. From the perspective of the participants, the studies would be similar. But from our end, we will be able to pace and regulate the exchanges with full control. The risk here was that if everyone took the same position and ended up being right, we would have to pay people a lot of money (whereas in the previous study, the winner always took the earnings of the loser). We had to hope that some people will be wrong sometime, or otherwise, we would go bankrupt quickly.

To be on the safe side we wrote a grant asking for a large sum of money that will sponsor the “worst-case scenario”. We got the money from Columbia University’s Tamer for Social Enterprise. And on we went.

It was not cheap. We had to pull strings, bite our nails, and also demand high commitment from our participants (we sent them daily reminders to make predictions daily, reached out to every individual who missed a bet, and provided endless technical support to participants; multiple Research Assistants were involved in this endeavor).

Alas, after weeks of effort, study 2 was complete. To reward us for the endless wait, the high expenses, and the sleepless nights, the results were double positive. Not only were we able to replicate the effects from study 1, we were also able to show the main effect we were after initially: when you use a representative sample (non-polarized individuals) in a study that makes people take positions about climate future, use the repeated feedback from winning/losing as an intrinsic motivation to shift perspectives and increase their engagement with climate topics - people become more concerned about climate change, more willing to support climate action, and more knowledgeable.

Encouraged by the learnings, we quickly wrote the results and resubmitted our work to Nature. Nearly 3 years after the first submission. Weeks later this paper was accepted.

As soon as the paper was accepted, the interest from the world started to materialize. People whose help we sought in creating bets (some who, by now, became part of the Administration; i.e., Gavin Schmidt) reached out with an interest in looking for ways to turn the academic work into a global functioning market. The reviewer who pushed us to redo the study with stronger control recommended that we write an additional “Policy Brief” paper to help legislators understand the value of the results to their domain. Journalists who learned about the work in various stages of its efforts asked if they could be the first to write about it. And others who supported us along the journey came together to think of the next steps.

Even one paper that, at some point seemed to us like it might be “scooping” us (a theoretical paper that spoke about the potential of the climate prediction market) ended up being instrumental in facilitating our work by educating some readers about the use-cases of climate prediction markets, and strengthening the understanding of the power of the works.

We are quite satisfied with the outcome. After eight years of effort, it was nice to see that our work has positive impact. But it is not the end.

It has not escaped our notice that there are many aspects of the work that need to be tested further. Our work in climate change is only the beginning. As the world becomes more polarized and less fact-driven, the power of prediction markets to solve some of today's alternative-realities ailments is enormous. A few of the obvious benefits are:

  • These markets allow people to transact in domains they do not see eye to eye on because they first agree on the arbiter of the bet (i.e., before we begin, we both agree that if, say, NASA declares that the temperatures increased by 1 degree, then you give me your money) and only then initiate the wagers. This reduces the risk of future disputes about the outcomes.
  • By virtue of their dynamic nature, these markets allow people to update their views as more information emerges (rather than static “horse-race” bets that do not change). This also means that new information is manifested in the value of a market and can be used to inform decisions.
  • These markets allow policymakers to gauge people’s actual views at any given moment (as an ultimate polling mechanism) where instead of stated opinions, people stake money on their views, making those seemingly more reliable.
  • One can make money from being smart/accurate.
  • The markets create a way to infuse private money into a climate domain, such that people find ways to invest their income in the future (with the right subsidies this could become a valid global financial instrument)
  • We allow people to change their views intrinsically — not by forcing views upon them or using expensive efforts that are often polarizing.

All of those suggest that this work has the potential to be a game-changer in how we address some of the challenges of our time.

Finally, we note that even if our ambitious goals remain unattained, there are some ways by which this work could be instrumental in helping climate science thrive: we term this “the win-win argument for climate prediction markets”.

If I (a person truly concerned about climate change) take a bet against a vocal climate denier (say, Charles Koch, CEO of Koch Industries) regarding climate outcomes (say, whether the water level in Miami will to a height that will make some areas of the city uninhabitable) and stake, say, $1,000,000 on that bet (with Mr. Koch putting his money where his mouth is and claiming that climate science is wrong and the water will not rise), once we both agree on a referee, the bet is on. In ten years, that referee determines if I was right or if Mr. Koch was right (presumably, if I am right, we would, unfortunately, know about it earlier).

If Mr.Koch took the bet it is ostensibly already a win for me. If I am right and win the bet - I can use the earnings to attempt and mitigate some of the climate outcomes. It won't help Miami at this stage, but it might help build better safeguards against hurricanes in New Orleans. If I lose and climate change not happening as Mr. Koch claims - I am equally happy. That is, I am happy to pay for the current climate models to be wrong.

Greta Thunberg will thank me.

Please sign in or register for FREE

If you are a registered user on Research Communities by Springer Nature, please sign in