Forecasting Newsletter: October 2021
Polymarket under investigation, IPCC climate prediction track record kind of ok.
Highlights
Polymarket is being investigated by the CFTC
David Friedman looks at the track record of IPCC predictions
Metaculus solves long-lasting UX problem: it now allows more expressive distributions.
Index
Prediction Markets & Forecasting Platforms
Job Board
In the News
Blog Posts
Long Content
Browse past newsletters here, or view this newsletter on the EA forum here. If you have a content suggestion or want to reach out, you can find me on Twitter.
Prediction Markets & Forecasting Platforms
Polymarket
Polymarket is being investigated by the US Commodity Futures Trading Commission (a). Back in 2008, several Nobel Prize winners and other academics called on US agencies to clarify and establish regulations on prediction markets (a). So at this point, I feel it's on the US Securities Exchange Commission and the Commodity Futures Trading Commission to have regulated this sooner, not on Polymarket not to have acted somewhat unilaterally.
Otherwise, Polymarket has continued to make more incremental usability improvements which make it more convenient for users to trade, add and withdraw funds from the site.
Metaculus
"nobody freak out but now you can asymmetrically adjust the tails of your logistic curves on continuous input questions on Metaculus" — @casens
SimonM (a) kindly curated the top comments from Metaculus this past August (a). They are:
SimonM (a) calculates a market-implied probability of US default
Matthew_Barnett (a) bets Robin Hanson that AGI will arrive before ems (and shares his logic)
PeterWildeford (a) goes long on Trump
SimonM (a) shares the historic General Social Survey data on sexlessness in young people
NunoSempere (a) lays out his logic for the rate of new donors appearing in the EA space
Metaculus held a panel discussion on its collaboration with the Virginia Health Department (a):
Since March 2020, Metaculus has provided forecasting and modeling resources to public health professionals as they've made crucial decisions in tracking and combating COVID-19. These efforts include the ongoing Keep Virginia Safe and the recently concluded Virginia Lightning Round forecasting tournaments, which were developed in partnership with the Virginia Department of Health and the University of Virginia Biocomplexity Institute, and were designed to enhance COVID-19 modeling efforts while contributing to the ongoing public health policy conversation in Virginia.
I feel that there is a disconnect between the "crucial decisions" and the very small prize pool of $1,000 given to forecasters (a). The moment that Metaculus' questions influence public health policy in Virginia at all, their Department of Health's willingness to pay should go through the roof.
In collaboration with Rethink Priorities (and with Michael Aird in particular), Metaculus has started short (a) and long-term (a) nuclear risk tournaments.
Good Judgment
The best comments on Good Judgment Open for October, as curated by myself, were:
Kogo (a) mentions that neither Europe nor China have the incentive to push for their Comprehensive Agreement on Investment in the current political standoff.
borisn outlines why the Democrats will most likely loose the House of Representatives (a) but retain control of the Senate (a), based on historical frequencies.
dada (a) wonders why the Good Judgment Open crowd assigns such a low probability to Iran and the US rejoining the JCPOA by the end of this year.
TerrySmith (a) mentions that Good Judgment Open seems to have beaten PredictIt on the 2020 election forecasts, so looking at betting odds might not be that informative for the Good Judgment Open crowd when discussing the upcoming election in Virginia.
belikewater (a) mentions a declassified intelligence report on COVID (a) which turned out to be inconclusive.
Good Judgment Open begins the In The News (a) and Dubai Future Experts (a) challenges.
Odds and ends
Futuur (a) is a prediction markets platform that recently came into my radar. They allow Americans to participate with a play-money currency, and the rest of the world to trade in dollars and in a variety of cryptocurrencies. They seem legitimate, and have been running a version of their site since 2017. But I would advise some caution: because all of their crypto-currencies go into the same pool, Futuur could be on the hook if the price of any one cryptocurrency moves too much too fast.
Reddit expands their prediction functionality (a) (original sources: 1 (a), 2 (a)). I've also become aware of a prediction community of Reddit at r/Predictor (a). With 14.4k members on that community alone, Reddit might just have become one of the biggest prediction platforms around, almost without even trying. h/t @marshallk (a).
Augur continues to focus on sports betting on Polygon with Augur Turbo, and saw upwards of $1M in trading volume, most likely because of the influx of subsidized liquidity (a).
CSET-Foretell continues to make progress (a) on their campaign around the future of the relationship between the US Department of Defense and US tech companies (a). They will have an event discussing their preliminary findings on the 10th of November (a).
Hedgehog Markets concluded their first two competitions, and will launch new ones, still focused on crypto and sports. Kalshi has also made a few improvements on their desktop webpage, and added some range markets, such as this one (a) on Chicago temperatures.
Forecasting Job Board.
The Perry World House (PWH) team at the University of Pennsylvania is looking for a Program Manager for the Future of the Global Order (a) because the current holder of the position is joining Founders Pledge. PWH has been "doing lots of work on implementing probabilistic forecasting methods in the U.S. government, and the person taking this job would likely continue work on those issues". One particularly high-quality piece of work by PWH previously mentioned in this newsletter was Keeping Score: A New Approach to Geopolitical Forecasting (a).
The Global Priorities Institute is looking for a Research Assistant (a) to aid its investigation into making forecasting a core research area. They are offering £17.48 ($23.85, 20.62€) per hour.
Metaculus is searching for "analytical storytellers" (a) on a rolling basis, paying around $0.3 per word ("essays are compensated at $300 each and at $25 per forecast question, with additional compensation awarded for especially high-quality essays attracting a significant readership")
North Dakota is looking for economic forecasting consultants (a). Although the offer seems to be aimed at individual consultants, I feel that it would also be interesting for forecasting platforms/prediction markets to apply.
Blog Posts
Charles Dillon of Rethink Priorities and SimonM look at How does forecast quantity impact forecast quality on Metaculus? (a). More forecasters increase forecast quality, but the effect is small beyond 10 or so forecasters.
One possible driver of this effect could be Metaculus allowing up to 10 forecasters to meaningfully coordinate in the public comments section, but not much beyond that.
David Friedman looks at whether the past IPCC temperature projections/predictions have been accurate? (a)
The predictions look better now than they did in 2014, high three times out of four, low once, and only once has actual warming been below the predicted range. They are still running a little high but the results look consistent with random error. That makes it at least possible that the IPCC researchers are now modeling the climate system well enough to produce reasonable estimates of its future behavior.
Jaime Sevilla writes about his current best guess on how to aggregate forecasts (a):
In Learning from our (the USA's) defeat (a) Tanner Greer of The Scholar's Stage looks at the leadership team of the second Bush's administration. It seems very much worth reading in terms of improving one's models of the world.
A Salesforce blogpost (a) advertises the wonders of cloud-based enterprise resource planning solutions (such as Salesforce itself.) Nonetheless, they still know what they are talking about.
In the News
Quartz covers the shutdown of Facebook's Forecast (a) in more depth.
FiveThirtyEight (a) writes a data-driven analysis of Biden's approval ratings.
US car sales are expected to plummet (a) due to chip shortage. I keep seeing this term "chip shortage", but it seems to me that this is more of a "supply chain mismanagement" issue because chips alone can't really make up that high a relative proportion of a vehicle's prize. Not also that Tesla doesn't seem to be affected by this shortage.
Warmer-than-normal temperatures could help save Americans on home heating costs, which could be elevated this year due to high energy prices (a), reports CNN. I find this curious because I'd expect CNN to avoid mentioning anything that could suggest that climate change is not unalloyedly negative. Still, the article doesn't mention the impact of Biden on energy prices, nor climate change directly.
Sephora, a beauty products brand, integrates AI more into its forecasting and replenishment software (a). To be clear, this is just business as normal, but it still feels like the kind of thing which is more likely in a world with short AI timelines (a).
Long Content
Issues with Futarchy (a) compiles possible failure modes with a governance model proposed by Robin Hanson where decisions would be made based on prediction markets.
Note: Due to EA Global (a), I now have a backlog of forecasting effort posts (a) posted during October. They will be incorporated into the next edition of this newsletter.
Note to the future: All links are added automatically to the Internet Archive, using this tool (a). "(a)" for archived links was inspired by Milan Griffes (a), Andrew Zuckerman (a), and Alexey Guzey (a).
Are you Alex Lawsen? The Alex Lawsen?
— Anonymous, EA Global 2021.