> being in that court might not be what you want to be doing.
and being in that court might change your behavior/incentives in a way that you don't reflectively endorse; it might lead you to do more of what you don't want to be doing.
I hadn't thought about the second order effect, but seems important. It could also be positive though, if the court has values and mores that you want to emulate. Seems 50/50
Platform economics are hard. I've been floating a prediction platform, but couldn't get it funded because potential funders were somewhat afraid of this two-sided element in platforms. You need to bring together both the forecasters and the people asking the questions. And early/existing platforms often take a lot of market share and mindspace, which makes it even harder for new entrants.
The argument I've heard even more often is that if you forecast politics, you would be in some sort of bull's eye. Politicians don't like to be shown their odds. More to the point, every market about people's odds to win some race (CEO, politicians etc) can be interpreted as assassination markets, around which there is understandable legal uncertainty.
The return on investment case is hard to make. I had (maybe mistakenly) said I'd use a nonprofit mantle for the company, in which investors would not get equity but bonds. That's hard to pull off in the beginning. And people interested in funding equity want the promise of a big, scalable ROI. I could not make that promise for a prediction market, seeing the cashburn and low user base of the existing competitors.
The most promising ways to promise a cash return is that you're collecting people's bet money and they won't withdraw it for a while, so you might actually invest the money parked on the platform behind user's backs. Some funds on an open prediction market won't ever be withdrawn, and could even be taken over by the company after some period of inactivity of 5-10y.
If you're into B2B sales, the companies will balk at the sustained extra effort required of staff to keep at making forecasts for years, forever. They prefer to buy from small-overhead cheap proven teams like yours or Michael's Swift. Key to your profitability is the very low overhead you have, but this way you can't scale (in the sense an investor would be interested in) beyond comfortable returns for your team.
This makes a lot of sense, thanks for sharing. I'd be curious to hear your thoughts about implications for platforms that do exist already, if you have any? Like, do these factors greatly reduce the likelihood of existing companies raising further VC funds? Should people be betting on the platforms that already exist? Or do you think this is your own skill issue and doesn't generalize?
My view is that the platforms that exist (Polymarket, Kalshi, Predictit, Metaculus, Insight, GJO, Infer, Hypermind, Manifold) are the Myspaces. They are pre-paradigmatic in the sense that none of them has broad appeal. Someone will therefore come and disrupt them, or perhaps disruption will come from the sites themselves (e.g. Manifold is trying very hard to self-disrupt). But someone trying for more of the same will
As for myself, it was less a skill and more an opportunity cost issue. As you wrote above, I've mostly moved on from forecasting because I have a family and a well paid career, so my environment rather strongly opposed me taking the jump. The timing was not right for me. That said I'm not sure what I would have added with my site that doesn't exist on the others, except it would have been in German.
I thought a bit more about disruption in that space. My conclusion was that the only way to really get big is to have a vehicle that sustains attention in our forecasts. The Base Rate Times should be on the news (if not) every day (then quite often). Think CNN, part of the Tagesschau in Germany or the Telediario in Spain. Like the weather forecast.
That would be a win-win, too. On the one hand, it would sustain more hype for the forecasting system, bringing in many more forecasters. And on the other hand, it would improve the news.
Great writeup, I think this captures the state well.
You mention Kalshi in your section on regulation, but having raised $30M in 2021, slightly larger (?) than the total funding OP has put into forecasting, its success or failure will be a landmark update in the space of forecasting affecting the world, I think.
One thing you don't mention is that leaving managerial incentives aside, there's huge value for corporates to use these mechanisms much more than they do. But a fundamental reason they often don’t is that 'spin' is today a huge part of most important decisions and forecasting competitions take the 'narrative' out of the hands of the organisation and put it in the hands of the competitors on the platform.
To your point about Samotsvety doing well recently, here's a link to the results of the 2023 ACX Forecasting Contest: https://www.astralcodexten.com/p/who-predicted-2023
and here's a link with additional details on my take regarding our participation: https://abstraction.substack.com/p/2023-acx-forecast-contest-thoughts
For readers passing by, the entry was more Jonathan Mann+ rather than Samotsvety proper
Does anyone want to explain to me like I'm five what the practical applications of forecasting might be, besides the weather?
Here are two possible ones:
- Improve personal decisions. Get better jobs, better romantic partners, live in better locations, earn more throughout lifetime
- Improve institutional decisions. Prioritize better, allocate ressources better, do the things in order of their probability of success times value.
Here is a specific example from my forecasting group: https://forum.effectivealtruism.org/posts/2nDTrDPZJBEerZGrk/samotsvety-nuclear-risk-update-october-2022 . You might also just want to browse the homepage of, say, Metaculus https://www.metaculus.com/home/ and see if anything strikes you as actionable.
Thank you -- highly informative.
> being in that court might not be what you want to be doing.
and being in that court might change your behavior/incentives in a way that you don't reflectively endorse; it might lead you to do more of what you don't want to be doing.
I hadn't thought about the second order effect, but seems important. It could also be positive though, if the court has values and mores that you want to emulate. Seems 50/50
Platform economics are hard. I've been floating a prediction platform, but couldn't get it funded because potential funders were somewhat afraid of this two-sided element in platforms. You need to bring together both the forecasters and the people asking the questions. And early/existing platforms often take a lot of market share and mindspace, which makes it even harder for new entrants.
The argument I've heard even more often is that if you forecast politics, you would be in some sort of bull's eye. Politicians don't like to be shown their odds. More to the point, every market about people's odds to win some race (CEO, politicians etc) can be interpreted as assassination markets, around which there is understandable legal uncertainty.
The return on investment case is hard to make. I had (maybe mistakenly) said I'd use a nonprofit mantle for the company, in which investors would not get equity but bonds. That's hard to pull off in the beginning. And people interested in funding equity want the promise of a big, scalable ROI. I could not make that promise for a prediction market, seeing the cashburn and low user base of the existing competitors.
The most promising ways to promise a cash return is that you're collecting people's bet money and they won't withdraw it for a while, so you might actually invest the money parked on the platform behind user's backs. Some funds on an open prediction market won't ever be withdrawn, and could even be taken over by the company after some period of inactivity of 5-10y.
If you're into B2B sales, the companies will balk at the sustained extra effort required of staff to keep at making forecasts for years, forever. They prefer to buy from small-overhead cheap proven teams like yours or Michael's Swift. Key to your profitability is the very low overhead you have, but this way you can't scale (in the sense an investor would be interested in) beyond comfortable returns for your team.
This makes a lot of sense, thanks for sharing. I'd be curious to hear your thoughts about implications for platforms that do exist already, if you have any? Like, do these factors greatly reduce the likelihood of existing companies raising further VC funds? Should people be betting on the platforms that already exist? Or do you think this is your own skill issue and doesn't generalize?
My view is that the platforms that exist (Polymarket, Kalshi, Predictit, Metaculus, Insight, GJO, Infer, Hypermind, Manifold) are the Myspaces. They are pre-paradigmatic in the sense that none of them has broad appeal. Someone will therefore come and disrupt them, or perhaps disruption will come from the sites themselves (e.g. Manifold is trying very hard to self-disrupt). But someone trying for more of the same will
As for myself, it was less a skill and more an opportunity cost issue. As you wrote above, I've mostly moved on from forecasting because I have a family and a well paid career, so my environment rather strongly opposed me taking the jump. The timing was not right for me. That said I'm not sure what I would have added with my site that doesn't exist on the others, except it would have been in German.
I thought a bit more about disruption in that space. My conclusion was that the only way to really get big is to have a vehicle that sustains attention in our forecasts. The Base Rate Times should be on the news (if not) every day (then quite often). Think CNN, part of the Tagesschau in Germany or the Telediario in Spain. Like the weather forecast.
That would be a win-win, too. On the one hand, it would sustain more hype for the forecasting system, bringing in many more forecasters. And on the other hand, it would improve the news.
> but better than the $1k it was a [check period]
Pending todo item :)
Whoops, fixed
Great writeup, I think this captures the state well.
You mention Kalshi in your section on regulation, but having raised $30M in 2021, slightly larger (?) than the total funding OP has put into forecasting, its success or failure will be a landmark update in the space of forecasting affecting the world, I think.
Cheers, agree
One thing you don't mention is that leaving managerial incentives aside, there's huge value for corporates to use these mechanisms much more than they do. But a fundamental reason they often don’t is that 'spin' is today a huge part of most important decisions and forecasting competitions take the 'narrative' out of the hands of the organisation and put it in the hands of the competitors on the platform.
This is a good point. But it's one specific hypothesis, and one could think of other reasons why prediction markets fail or don't fail in the corporate setting (https://forum.effectivealtruism.org/posts/dQhjwHA7LhfE8YpYF/prediction-markets-in-the-corporate-setting)
Thanks for the response and the reference.
Thanks for this great overview. I gave you (and Metaforecast) a shout-out here: https://braff.co/advice/f/things-to-do-in-greater-nerdville