The use of p-values and hypothesis testing is ubiquitous across science. Everyone remembers the rules they learnt in their first statistics courses; 0.05 is the magic cut-off for statistical significance. Statistical evidence with p-values below 0.05 are accepted – models with p-values higher than this threshold are discarded. However, this statistical practice has come…
If you’re used to financial markets, you may wonder where the “Buy” and “Sell” buttons are. Most markets focus on buying and selling, and determine the price (probability) under the hood. We use the same formula, but focus on the probability (price), determining shares under the hood. To the right of the image below, you…
The social and behavioral sciences, some say, have a “replication crisis.” When researchers attempt to reproduce results of published papers, sometimes the original conclusions do not hold up. Replication studies are conducted using the same methods and standards laid out in the published paper. They use new participants and often a larger sample size. Researchers…
In a number of past research projects, we have used prediction markets to elicit information on the replicability of research studies. In these markets, participants traded contracts with payoffs tied to the outcome of replications in large-scale replication projects, and thereby created forecasts – similar to bettors in sports betting markets who create forecasts by…
Anna Dreber Almenberg, part of Replication Markets’ experiment design team, is the Johan Björkman Professor of Economics at the Stockholm School of Economics. She took time recently to answer a few questions about her interest in the field of research replication. What got you thinking about replications and markets? For me it all started when…
If you’ve just read What is a high-quality replication, you know that our replications will be high-power — usually much higher than the original study — but not quite the “100 replications” ideal set out by DARPA: Assume 100 replications of this study were performed, and a weighted average of their results (i.e., a meta-analysis) was…
From the replication crisis to forecasting to prediction markets, the Replication Markets project covers a broad range of intellectual territory. Explore our recommended reading to learn more.
