The resolutions for the prediction market for the 400 preprints related to COVID-19 have been finalized, and prizes have been calculated. Winners have been notified by email at the address registered with the username on the Replication Markets platform. Prizes must be claimed by March 18, 2022. (Though if you could do so by Feb…
Review In late 2020 we asked forecasters to predict the fate of 400 popular COVID-19 preprints posted to bioRxiv and medRxiv between January and September of that year. For each we asked: Would it be published within a year of posting, and if so, would it be in a high-impact (JIF≥10) journal? [No / High / Low]…
In November of 2020, Replication Markets ran a prediction market for 400 preprints related to COVID-19. Categories included Epidemiology, Infectious Diseases, Immunology, Microbiology, and Public and Global Health. Participants had the opportunity to bid on four questions for each of the 400 preprints: Citation ranking of the preprint one year from the preprint publication date.…
When Replication Markets launched in 2019, we could not have expected the global interruptions that affected our project timeline. Finally, we are ready to resolve and pay out our markets! One silver lining: in waiting to announce results, we have more resolutions to pay out. More resolutions means a larger sample size to answer the…
We expected to announce prizes this week, but co-pilot efforts just ran slowly and after verifying Rounds 1-11, today we found a small discrepancy in our Round 0 (meta-claims) calculations that wasn’t sorted by end of day. This will certainly be embarrassingly obvious with a fresh look on Monday. In the meantime, we can say…
The Replication Markets team expects to begin prize announcements by 8-NOV. We are double-checking points distribution and prize values, and will do a last check that there are no new replication results we could use. So far we have 126 forecasters who will receive prizes for Rounds 1-10. Winners will receive emails, the winners list…
SCORE Markets (Rounds 1-10, plus 0 & 11): I waited as long as possible to maximize the number of completed replications, but we now have about two months to resolve claims, pay prizes, and wrap up. There are now max 88 qualifying claims: better than in February, but less than we would like. It is…
Hello forecasters! A quick status update Survey prizes paid out as planned, but we’re still awaiting final December resolutions to let us pay out the main SCORE markets. COS says we should have it shortly. There will be far fewer resolutions than expected, but we cannot wait a year for pandemic-halted experiments. Updated what-to-expect:• We…
Replication Markets recently estimated replication probabilities for 100 COVID-19 studies from the social and behavioral sciences. We don’t have all the results yet, but let’s see what our forecasters thought. (Note: We just launched another COVID-19 project, we hope you’ll sign-up to participate or learn more). Reviewing our Previous COVID-19 Forecasts Big picture ~75% of…
Surrogate scoring rules (SSR) is the method we used to score forecasters for accuracy in our survey. We encourage all forecasters to watch our short video about the method. This document has three sections: (1) Overview, (2) Implementation, (3) Example. SSR Overview Surrogate scoring rules enable us to score forecasts’ accuracy using strictly proper scoring…
