Why bets aren't a good measure of beliefs
Matthew Martin 10/03/2013 05:43:00 PM
|Example of a conditional forecast.|
An interesting question is whether such people actually believe their own claims, since they can costlessly spew predictive garbage. Perhaps their goal is to try to persuade people to go along with their policy preferences which they value for other reasons. There is a strain of thought that has been tossed around, mostly by libertarian-leaning economists, that we can cut out the BS by forcing commentators to make bets about their claims. If the commentator doesn't actually believe QE will cause hyperinflation--the thinking goes--then he won't be willing to bet money that it will. The implication here is that we should be dismissive of any commentator who makes a prediction but refuses to bet on it.
While I appreciate the effort to cut through the BS, this idea that betting can reveal people's beliefs is utter BS. Noah Smith has a post offering a few reasons why, but he misses the most important one: you can't bet on conditional forecasts.
What made me think of this was the recent release of the IPCC's fifth report on climate change. Suppose I challenged a climate change denier to a bet over whether global temperatures will continue to rise. Specifically whether the average global surface temperatures will be more than 1 degree Celsius higher in the 2080s than the 2010s (an event considered "extremely likely" by the IPCC's report). Obviously that's a long-range forecast, but assume for the moment that we'll both still be alive and able to pay by then.
Does this bet force either of us to clarify our beliefs about global warming? NO! In fact, while I believe in global warming, and agree completely with the IPCC's forecast, I don't actually think temperatures in 2080 will be that much higher. That's not a contradiction at all--the IPCC's forecast is conditional on no policy action to reduce the concentration of greenhouse gases in the atmosphere. Call me an optimist, but I don't think that will prove true: sometime before 2080, I think policy makers will successfully curb carbon emissions and we will clock in well below the IPCC's forecast. As the new report shows, in just the past five years we've made huge progress on that front. My point here is that how a rational person would bet depends not just on his belief in the conditional forecast--in this case, that with no policy change temperatures will rise--but also on his belief in a variety of extraneous information irrelevant to the question. Global warming is real, and we will find a way to avert it.
This isn't a rare problem. Some version of this problem presents itself in any policy-related forecast. Take the CBO's deficit forecasts. When the CBO forecasts a large deficit, congress takes action to reduce it. Because of the policy change congress made in response to the accurate deficit forecast, the forecast turned out to be wrong.
Here is an extremely concrete example. Suppose NASA predicts an asteroid will obliterate the earth in 2030, and persuades congress to launch some high-yield missiles to throw it off course. The fact that, come 2030, the asteroid actually ends up missing the earth is not evidence that NASA's original forecast was inaccurate. It's just that the forecast was conditional on no policy response. And you can't resolve a conditional bet on NASA's prediction, because you will never be able to observe the counter-factual scenario in which no policy response was taken.
The bottom line here is that all good policy forecasts should be counter-factual. They must assume something that is not true--that policy makers do not change policies--in order to yield any useful advice on what policies should be changed. And that, of course, means that all good policy forecasts will be wrong. Wrong, wrong, wrong. Wrong, even if they are right.