We have big changes to announce! Read to learn about…
- simpler definition of replication
- more markets paying out
- simpler surveys
Simpler Definition of Replication
SCORE has adopted a simpler, unified definition of replication:
That required some changes from us. Starting in Round 6, Replication Markets will no longer distinguish between “data replication” and “direct replication.”
Studies selected for replication will use the most appropriate type, and all will count for market resolutions.
More Markets Pay Out!
The new definition of replication increases the expected market replications from about 50 in Rounds 1-5 to about 125 in Rounds 6-10.
Survey-Takers, rejoice! The new definition also means we can simplify our surveys!
Surveys now have only 4 questions (rather than 7). In essence, they ask:
- What is the probability that a high-power replication would succeed?
- What fraction of participants will give an estimate larger than 50% on the previous question?
- How plausible is this claim?
- Is there anything else we should know?
With the change, the other three questions became redundant. We considered keeping them anyway, but they were providing little information: Old Q3 was 93% correlated with old Q1, with a slope of 1. (We will make this data available to explore after claims resolve.)