The Replication Markets team expects to begin prize announcements by 8-NOV.
We are double-checking points distribution and prize values, and will do a last check that there are no new replication results we could use. So far we have 126 forecasters who will receive prizes for Rounds 1-10.
Winners will receive emails, the winners list will be posted on this blog, and you’ll see information in our Twitter feed and Reddit channel.
Until then, enjoy this general preview! (Final numbers subject to change.)
Reminder about the Replication Markets SCORE structure:
Round 0: Meta-questions – still working on those resolutions, as they depend on others
Rounds 1-5: About 1500 claims, only direct replications count towards market resolutions.
Rounds 6-10: About 1500 claims, both direct replications and data analytic replications count.
Round 11: 100 COVID-related preprints, direct and data analytic replications count
We ignore computational reproductions. Rounds 1-10 have had 87 replications so far:
- 33 data analytic replication results (15 successful, or 45%)
- 54 direct replication results (23 successful, or 43%)
We can use 66 of those for prizes (dropping data analytic replications in Rounds 1-5):
- 29 of these 66 claims successfully replicated
- 37 of these 66 claims did NOT replicate
(There was one more eligible replication, but RM forecasters never saw the claim because it was one of 19 judged inappropriate for betting on.)
These are the numbers most subject to change. At the moment, it seems 126 forecasters have net winning positions in R1-10.
- 108 users have 1, 2, or 3 winning positions
- Maximum number of winning positions is 15
Prizes are computed based on the proportion of points in a single Round.
- Maximum proportion of winning points is 100% !
- Median proportion of winning points is 13%
- Mean proportion of winning points is 23%