Forecasting Newsletter: October 2021
Polymarket under investigation, IPCC climate prediction track record kind of ok.
Polymarket is being investigated by the CFTC
David Friedman looks at the track record of IPCC predictions
Metaculus solves long-lasting UX problem: it now allows more expressive distributions.
Prediction Markets & Forecasting Platforms
In the News
Prediction Markets & Forecasting Platforms
Polymarket is being investigated by the US Commodity Futures Trading Commission (a). Back in 2008, several Nobel Prize winners and other academics called on US agencies to clarify and establish regulations on prediction markets (a). So at this point, I feel it's on the US Securities Exchange Commission and the Commodity Futures Trading Commission to have regulated this sooner, not on Polymarket not to have acted somewhat unilaterally.
Otherwise, Polymarket has continued to make more incremental usability improvements which make it more convenient for users to trade, add and withdraw funds from the site.
"nobody freak out but now you can asymmetrically adjust the tails of your logistic curves on continuous input questions on Metaculus" — @casens
Since March 2020, Metaculus has provided forecasting and modeling resources to public health professionals as they've made crucial decisions in tracking and combating COVID-19. These efforts include the ongoing Keep Virginia Safe and the recently concluded Virginia Lightning Round forecasting tournaments, which were developed in partnership with the Virginia Department of Health and the University of Virginia Biocomplexity Institute, and were designed to enhance COVID-19 modeling efforts while contributing to the ongoing public health policy conversation in Virginia.
I feel that there is a disconnect between the "crucial decisions" and the very small prize pool of $1,000 given to forecasters (a). The moment that Metaculus' questions influence public health policy in Virginia at all, their Department of Health's willingness to pay should go through the roof.
The best comments on Good Judgment Open for October, as curated by myself, were:
TerrySmith (a) mentions that Good Judgment Open seems to have beaten PredictIt on the 2020 election forecasts, so looking at betting odds might not be that informative for the Good Judgment Open crowd when discussing the upcoming election in Virginia.
Odds and ends
Futuur (a) is a prediction markets platform that recently came into my radar. They allow Americans to participate with a play-money currency, and the rest of the world to trade in dollars and in a variety of cryptocurrencies. They seem legitimate, and have been running a version of their site since 2017. But I would advise some caution: because all of their crypto-currencies go into the same pool, Futuur could be on the hook if the price of any one cryptocurrency moves too much too fast.
Reddit expands their prediction functionality (a) (original sources: 1 (a), 2 (a)). I've also become aware of a prediction community of Reddit at r/Predictor (a). With 14.4k members on that community alone, Reddit might just have become one of the biggest prediction platforms around, almost without even trying. h/t @marshallk (a).
CSET-Foretell continues to make progress (a) on their campaign around the future of the relationship between the US Department of Defense and US tech companies (a). They will have an event discussing their preliminary findings on the 10th of November (a).
Hedgehog Markets concluded their first two competitions, and will launch new ones, still focused on crypto and sports. Kalshi has also made a few improvements on their desktop webpage, and added some range markets, such as this one (a) on Chicago temperatures.
Forecasting Job Board.
The Perry World House (PWH) team at the University of Pennsylvania is looking for a Program Manager for the Future of the Global Order (a) because the current holder of the position is joining Founders Pledge. PWH has been "doing lots of work on implementing probabilistic forecasting methods in the U.S. government, and the person taking this job would likely continue work on those issues". One particularly high-quality piece of work by PWH previously mentioned in this newsletter was Keeping Score: A New Approach to Geopolitical Forecasting (a).
Metaculus is searching for "analytical storytellers" (a) on a rolling basis, paying around $0.3 per word ("essays are compensated at $300 each and at $25 per forecast question, with additional compensation awarded for especially high-quality essays attracting a significant readership")
North Dakota is looking for economic forecasting consultants (a). Although the offer seems to be aimed at individual consultants, I feel that it would also be interesting for forecasting platforms/prediction markets to apply.
Charles Dillon of Rethink Priorities and SimonM look at How does forecast quantity impact forecast quality on Metaculus? (a). More forecasters increase forecast quality, but the effect is small beyond 10 or so forecasters.
One possible driver of this effect could be Metaculus allowing up to 10 forecasters to meaningfully coordinate in the public comments section, but not much beyond that.
David Friedman looks at whether the past IPCC temperature projections/predictions have been accurate? (a)
The predictions look better now than they did in 2014, high three times out of four, low once, and only once has actual warming been below the predicted range. They are still running a little high but the results look consistent with random error. That makes it at least possible that the IPCC researchers are now modeling the climate system well enough to produce reasonable estimates of its future behavior.
In Learning from our (the USA's) defeat (a) Tanner Greer of The Scholar's Stage looks at the leadership team of the second Bush's administration. It seems very much worth reading in terms of improving one's models of the world.
In the News
US car sales are expected to plummet (a) due to chip shortage. I keep seeing this term "chip shortage", but it seems to me that this is more of a "supply chain mismanagement" issue because chips alone can't really make up that high a relative proportion of a vehicle's prize. Not also that Tesla doesn't seem to be affected by this shortage.
Warmer-than-normal temperatures could help save Americans on home heating costs, which could be elevated this year due to high energy prices (a), reports CNN. I find this curious because I'd expect CNN to avoid mentioning anything that could suggest that climate change is not unalloyedly negative. Still, the article doesn't mention the impact of Biden on energy prices, nor climate change directly.
Sephora, a beauty products brand, integrates AI more into its forecasting and replenishment software (a). To be clear, this is just business as normal, but it still feels like the kind of thing which is more likely in a world with short AI timelines (a).
Note to the future: All links are added automatically to the Internet Archive, using this tool (a). "(a)" for archived links was inspired by Milan Griffes (a), Andrew Zuckerman (a), and Alexey Guzey (a).
Are you Alex Lawsen? The Alex Lawsen?
— Anonymous, EA Global 2021.